[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:43:40 dist-test-slave-n326 krb5kdc[30052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:43:40 dist-test-slave-n326 krb5kdc[30052](info): set up 2 sockets
Aug 06 00:43:40 dist-test-slave-n326 krb5kdc[30052](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 00:43:42.254796 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.391s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:43:42 dist-test-slave-n326 krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:43:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:43:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:43:42.382125 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38109
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:38109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:42.502079 30071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:42.502300 30071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:42.502336 30071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:42.506503 30071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:43:42.506563 30071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:42.506582 30071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:42.506598 30071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:43:42.506613 30071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:43:42.511639 30071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:38109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:42.512549 30071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:42.513489 30071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:42.522253 30080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:42.522256 30079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:42.522256 30083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:42.522408 30071 server_base.cc:1034] running on GCE node
I20260806 00:43:42.523133 30071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:42.524443 30071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:42.525638 30071 hybrid_clock.cc:648] HybridClock initialized: now 1785977022525617 us; error 33 us; skew 500 ppm
Aug 06 00:43:42 dist-test-slave-n326 krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977022, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:42.528568 30071 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:43:42.529351 30071 webserver.cc:460] Webserver started at http://127.29.84.190:36273/ using document root <none> and password file <none>
I20260806 00:43:42.529903 30071 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:42.529991 30071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:42.530296 30071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:42.532193 30071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "a0ddd4f851bd42b8833b083a786c94c0"
format_stamp: "Formatted at 2026-08-06 00:43:42 on dist-test-slave-n326"
server_key: "1b52ee5115d77ee4c5832be8070808fa"
server_key_iv: "43061e1f957584c2db086609dae1754c"
server_key_version: "encryptionkey@0"
I20260806 00:43:42.532734 30071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "a0ddd4f851bd42b8833b083a786c94c0"
format_stamp: "Formatted at 2026-08-06 00:43:42 on dist-test-slave-n326"
server_key: "1b52ee5115d77ee4c5832be8070808fa"
server_key_iv: "43061e1f957584c2db086609dae1754c"
server_key_version: "encryptionkey@0"
I20260806 00:43:42.536382 30071 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:43:42.538652 30094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:42.539811 30071 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:43:42.539954 30071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "a0ddd4f851bd42b8833b083a786c94c0"
format_stamp: "Formatted at 2026-08-06 00:43:42 on dist-test-slave-n326"
server_key: "1b52ee5115d77ee4c5832be8070808fa"
server_key_iv: "43061e1f957584c2db086609dae1754c"
server_key_version: "encryptionkey@0"
I20260806 00:43:42.540059 30071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:42.557327 30071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:42.560537 30071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:42.560673 30071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:42.567911 30071 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:38109
I20260806 00:43:42.567981 30169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:38109 every 8 connection(s)
I20260806 00:43:42.568836 30071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:43:42.571522 30173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:42.576849 30173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0: Bootstrap starting.
I20260806 00:43:42.578135 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30071
I20260806 00:43:42.578274 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:43:42.579300 30173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:42.579617 30034 external_mini_cluster.cc:1428] Setting key 3178c47b3ffd54ceefa901c22d2222d0
I20260806 00:43:42.580057 30173 log.cc:824] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:42.582132 30173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0: No bootstrap required, opened a new log
I20260806 00:43:42.586239 30173 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ddd4f851bd42b8833b083a786c94c0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38109 } }
I20260806 00:43:42.586457 30173 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:42.586545 30173 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ddd4f851bd42b8833b083a786c94c0, State: Initialized, Role: FOLLOWER
I20260806 00:43:42.586962 30173 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ddd4f851bd42b8833b083a786c94c0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38109 } }
I20260806 00:43:42.587137 30173 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:43:42.587212 30173 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:43:42.587302 30173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:42.588382 30173 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ddd4f851bd42b8833b083a786c94c0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38109 } }
I20260806 00:43:42.588708 30173 leader_election.cc:302] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0ddd4f851bd42b8833b083a786c94c0; no voters:
I20260806 00:43:42.589018 30173 leader_election.cc:288] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:43:42.589102 30179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:42.589277 30179 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [term 1 LEADER]: Becoming Leader. State: Replica: a0ddd4f851bd42b8833b083a786c94c0, State: Running, Role: LEADER
I20260806 00:43:42.589543 30179 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ddd4f851bd42b8833b083a786c94c0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38109 } }
I20260806 00:43:42.590222 30173 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:43:42.590755 30179 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0ddd4f851bd42b8833b083a786c94c0. Latest consensus state: current_term: 1 leader_uuid: "a0ddd4f851bd42b8833b083a786c94c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ddd4f851bd42b8833b083a786c94c0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38109 } } }
I20260806 00:43:42.590852 30179 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:43:42.591790 30180 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0ddd4f851bd42b8833b083a786c94c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ddd4f851bd42b8833b083a786c94c0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38109 } } }
I20260806 00:43:42.591965 30180 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0 [sys.catalog]: This master's current role is: LEADER
W20260806 00:43:42.595225 30206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:43:42.595307 30206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:43:42.595366 30187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:43:42.596423 30187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 00:43:42 dist-test-slave-n326 krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:42.602099 30187 catalog_manager.cc:1347] Generated new cluster ID: 15d45956fb3e4375840bc006e8064248
I20260806 00:43:42.602208 30187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:43:42.604844 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:42.582780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.29.84.190:38109)
0806 00:43:42.583218 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:42.583233 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:42.584420 (+ 1187us) server_negotiation.cc:407] Connection header received
0806 00:43:42.587199 (+ 2779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:42.587218 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:42.587581 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:42.587961 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:42.590298 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:42.592679 (+ 2381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:42.593879 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:42.594383 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:42.594574 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:42.598579 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:42.598586 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:42.598610 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:42.598641 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:42.600844 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:42.601531 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:42.601539 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:42.601544 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:42.601736 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:42.602203 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:42.602206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:42.602208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:42.602798 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:42.602993 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:42.603624 (+ 631us) server_negotiation.cc:299] Negotiation successful
0806 00:43:42.603913 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":823,"server-negotiator.queue_time_us":305,"thread_start_us":134,"threads_started":1}
I20260806 00:43:42.612207 30187 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:43:42.613270 30187 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:43:42.621706 30187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0ddd4f851bd42b8833b083a786c94c0: Generated new TSK 0
I20260806 00:43:42.622457 30187 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:43:42.657790 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38109
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:42.806917 30214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:42.807240 30214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:42.807340 30214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:42.811223 30214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:42.811299 30214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:42.811391 30214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:43:42.816546 30214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38109
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:42.817467 30214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:42.818408 30214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:42.829182 30221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:42.829259 30220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:42.829152 30223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:42.829383 30214 server_base.cc:1034] running on GCE node
I20260806 00:43:42.829788 30214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:42.830363 30214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:42.831547 30214 hybrid_clock.cc:648] HybridClock initialized: now 1785977022831533 us; error 25 us; skew 500 ppm
Aug 06 00:43:42 dist-test-slave-n326 krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977022, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:42.834597 30214 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:43:42.835466 30214 webserver.cc:460] Webserver started at http://127.29.84.129:46861/ using document root <none> and password file <none>
I20260806 00:43:42.835999 30214 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:42.836102 30214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:42.836340 30214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:42.838392 30214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "7d8838c71fd34958b80be14e308af91f"
format_stamp: "Formatted at 2026-08-06 00:43:42 on dist-test-slave-n326"
server_key: "0af18c7a197ba723e6cc4a3e51731380"
server_key_iv: "baf30b075446dcf73e1d37179ca31734"
server_key_version: "encryptionkey@0"
I20260806 00:43:42.838964 30214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "7d8838c71fd34958b80be14e308af91f"
format_stamp: "Formatted at 2026-08-06 00:43:42 on dist-test-slave-n326"
server_key: "0af18c7a197ba723e6cc4a3e51731380"
server_key_iv: "baf30b075446dcf73e1d37179ca31734"
server_key_version: "encryptionkey@0"
I20260806 00:43:42.842711 30214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:43:42.845301 30233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:42.846474 30214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:43:42.846608 30214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "7d8838c71fd34958b80be14e308af91f"
format_stamp: "Formatted at 2026-08-06 00:43:42 on dist-test-slave-n326"
server_key: "0af18c7a197ba723e6cc4a3e51731380"
server_key_iv: "baf30b075446dcf73e1d37179ca31734"
server_key_version: "encryptionkey@0"
I20260806 00:43:42.846720 30214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:42.861882 30214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:42.864733 30214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:42.864975 30214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:42.865676 30214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:43:42.866570 30214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:42.866645 30214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:42.866710 30214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:42.866762 30214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:42.886390 30214 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:37723
I20260806 00:43:42.886520 30401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:37723 every 8 connection(s)
I20260806 00:43:42.887437 30214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:43:42.894425 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30214
I20260806 00:43:42.894512 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:43:42.894802 30034 external_mini_cluster.cc:1428] Setting key 20dba65033518d09cce660147b5939aa
Aug 06 00:43:42 dist-test-slave-n326 krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977022, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:42.903254 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:42.889528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:59827 (local address 127.29.84.190:38109)
0806 00:43:42.889678 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:42.889694 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:42.891109 (+ 1415us) server_negotiation.cc:407] Connection header received
0806 00:43:42.892257 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:42.892260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:42.892306 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:42.892378 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:42.893644 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:42.894186 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:42.895868 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:42.896198 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:42.896312 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:42.899507 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:42.899510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:42.899513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:42.899536 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:42.900949 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:42.901554 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:42.901561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:42.901566 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:42.901667 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:42.902011 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:42.902016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:42.902019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:42.902275 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:42.902407 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:42.902848 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 00:43:42.902992 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:43:42.903847 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:42.889956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38109 (local address 127.29.84.129:59827)
0806 00:43:42.890917 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:42.890975 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:42.892079 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:42.892532 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:42.892538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:42.892968 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:42.893520 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:42.893534 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:42.894326 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:42.894328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:42.895749 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:42.895760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:42.896359 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:42.896363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:42.896579 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:42.897190 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:42.897205 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:42.899331 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:42.901072 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:42.901077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:42.901097 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:42.901381 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:42.901771 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:42.901773 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:42.901774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:42.901901 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:42.902388 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:42.902393 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:42.902589 (+ 196us) client_negotiation.cc:769] Sending connection context
0806 00:43:42.902737 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 00:43:42.902951 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":185,"threads_started":1}
I20260806 00:43:42.905083 30402 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38109
I20260806 00:43:42.905314 30402 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:42.905757 30402 heartbeater.cc:511] Master 127.29.84.190:38109 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:43:42.907511 30116 ts_manager.cc:194] Registered new tserver with Master: 7d8838c71fd34958b80be14e308af91f (127.29.84.129:37723)
I20260806 00:43:42.908620 30116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:59827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:43:42.926244 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38109
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:43.078682 30407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:43.078897 30407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:43.078940 30407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:43.082808 30407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:43.082855 30407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:43.082911 30407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:43:43.088131 30407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38109
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:43.088955 30407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:43.089775 30407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:43.099254 30415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:43.099272 30414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:43.099290 30417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:43.099406 30407 server_base.cc:1034] running on GCE node
I20260806 00:43:43.099997 30407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:43.100549 30407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:43.101804 30407 hybrid_clock.cc:648] HybridClock initialized: now 1785977023101763 us; error 53 us; skew 500 ppm
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977023, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:43.105111 30407 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:43:43.105906 30407 webserver.cc:460] Webserver started at http://127.29.84.130:35961/ using document root <none> and password file <none>
I20260806 00:43:43.106379 30407 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:43.106433 30407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:43.106578 30407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:43.108675 30407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "7f184977c51c49af98b06ecbf0afd953"
format_stamp: "Formatted at 2026-08-06 00:43:43 on dist-test-slave-n326"
server_key: "ffeddefb52831bfba841a6012236001d"
server_key_iv: "7d74458622e534af55808d15c9bea7c1"
server_key_version: "encryptionkey@0"
I20260806 00:43:43.109195 30407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "7f184977c51c49af98b06ecbf0afd953"
format_stamp: "Formatted at 2026-08-06 00:43:43 on dist-test-slave-n326"
server_key: "ffeddefb52831bfba841a6012236001d"
server_key_iv: "7d74458622e534af55808d15c9bea7c1"
server_key_version: "encryptionkey@0"
I20260806 00:43:43.112828 30407 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:43:43.115166 30429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:43.116297 30407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:43.116389 30407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "7f184977c51c49af98b06ecbf0afd953"
format_stamp: "Formatted at 2026-08-06 00:43:43 on dist-test-slave-n326"
server_key: "ffeddefb52831bfba841a6012236001d"
server_key_iv: "7d74458622e534af55808d15c9bea7c1"
server_key_version: "encryptionkey@0"
I20260806 00:43:43.116464 30407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:43.130223 30407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:43.133399 30407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:43.133615 30407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:43.134235 30407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:43:43.134979 30407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:43.135026 30407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:43.135128 30407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:43.135159 30407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:43.154522 30407 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:42077
I20260806 00:43:43.154644 30594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:42077 every 8 connection(s)
I20260806 00:43:43.155457 30407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:43:43.162458 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30407
I20260806 00:43:43.162588 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:43:43.162858 30034 external_mini_cluster.cc:1428] Setting key d5c7f4d178a931d1826b8c2b081c2a37
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977023, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:43.170635 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:43.157323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:51101 (local address 127.29.84.190:38109)
0806 00:43:43.157479 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:43.157483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:43.158456 (+ 973us) server_negotiation.cc:407] Connection header received
0806 00:43:43.159361 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:43.159365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:43.159428 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:43.159524 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:43.160801 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.161340 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.162723 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.163035 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.163169 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:43.166206 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:43.166209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:43.166211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:43.166250 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:43.168301 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:43.168804 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:43.168811 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:43.168815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:43.168908 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:43.169303 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:43.169309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:43.169314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:43.169600 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:43.169712 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:43.170269 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 00:43:43.170405 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:43:43.171479 30597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:43.157653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38109 (local address 127.29.84.130:51101)
0806 00:43:43.158294 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:43.158332 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:43.159166 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:43.159692 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:43.159698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:43.160115 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:43.160658 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:43.160672 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.161487 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.161490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:43.162506 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:43.162516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.163216 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.163219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:43.163503 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:43.164142 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:43.164158 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:43.166045 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:43.168440 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:43.168444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:43.168454 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:43.168694 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:43.169038 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:43.169041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:43.169043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:43.169184 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:43.169720 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:43.169725 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:43.170027 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 00:43:43.170184 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 00:43:43.170451 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":123,"threads_started":1}
I20260806 00:43:43.172766 30595 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38109
I20260806 00:43:43.173061 30595 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:43.173694 30595 heartbeater.cc:511] Master 127.29.84.190:38109 requested a full tablet report, sending...
I20260806 00:43:43.174968 30116 ts_manager.cc:194] Registered new tserver with Master: 7f184977c51c49af98b06ecbf0afd953 (127.29.84.130:42077)
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:43:43.175897 30116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:51101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:43:43.197645 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38109
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:43.340183 30600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:43.340435 30600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:43.340478 30600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:43.344413 30600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:43.344460 30600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:43.344556 30600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:43:43.349928 30600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38109
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:43.350862 30600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:43.351804 30600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:43:43.362188 30600 server_base.cc:1034] running on GCE node
W20260806 00:43:43.362105 30609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:43.362131 30606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:43.362162 30607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:43.362763 30600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:43.363430 30600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:43.364614 30600 hybrid_clock.cc:648] HybridClock initialized: now 1785977023364602 us; error 26 us; skew 500 ppm
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977023, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:43.367424 30600 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:43:43.368131 30600 webserver.cc:460] Webserver started at http://127.29.84.131:36781/ using document root <none> and password file <none>
I20260806 00:43:43.368590 30600 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:43.368645 30600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:43.368789 30600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:43.370525 30600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "72f38b0950864d66b9740c016f3f1383"
format_stamp: "Formatted at 2026-08-06 00:43:43 on dist-test-slave-n326"
server_key: "c812d9cf8cca712e9f469747fd24d7da"
server_key_iv: "4854fca0003c417c542dde72c3d2a6f3"
server_key_version: "encryptionkey@0"
I20260806 00:43:43.370994 30600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "72f38b0950864d66b9740c016f3f1383"
format_stamp: "Formatted at 2026-08-06 00:43:43 on dist-test-slave-n326"
server_key: "c812d9cf8cca712e9f469747fd24d7da"
server_key_iv: "4854fca0003c417c542dde72c3d2a6f3"
server_key_version: "encryptionkey@0"
I20260806 00:43:43.374708 30600 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:43:43.377015 30617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:43.378123 30600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:43.378253 30600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "72f38b0950864d66b9740c016f3f1383"
format_stamp: "Formatted at 2026-08-06 00:43:43 on dist-test-slave-n326"
server_key: "c812d9cf8cca712e9f469747fd24d7da"
server_key_iv: "4854fca0003c417c542dde72c3d2a6f3"
server_key_version: "encryptionkey@0"
I20260806 00:43:43.378351 30600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:43.401832 30600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:43.404935 30600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:43.405131 30600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:43.405857 30600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:43:43.406725 30600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:43.406785 30600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:43.406849 30600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:43.406883 30600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:43.424467 30600 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:37053
I20260806 00:43:43.424803 30794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:37053 every 8 connection(s)
I20260806 00:43:43.425546 30600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:43:43.433882 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30600
I20260806 00:43:43.434006 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:43:43.434276 30034 external_mini_cluster.cc:1428] Setting key e238f3e5a6e05b04b56cbd6dd70efdf0
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977023, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:43.438721 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:43.427299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:44393 (local address 127.29.84.190:38109)
0806 00:43:43.427459 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:43.427463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:43.428251 (+ 788us) server_negotiation.cc:407] Connection header received
0806 00:43:43.429263 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:43.429266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:43.429314 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:43.429385 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:43.430781 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.431300 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.432484 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.432693 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.432766 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:43.435321 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:43.435325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:43.435327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:43.435352 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:43.436954 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:43.437481 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:43.437484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:43.437485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:43.437563 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:43.437848 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:43.437850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:43.437851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:43.438016 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:43.438092 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:43.438507 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 00:43:43.438598 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:43:43.439502 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:43.427580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38109 (local address 127.29.84.131:44393)
0806 00:43:43.428098 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:43.428138 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:43.429085 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:43.429572 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:43.429578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:43.430044 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:43.430640 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:43.430655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.431429 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.431431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:43.432370 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:43.432379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.432812 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.432814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:43.432980 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:43.433537 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:43.433552 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:43.435149 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:43.437098 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:43.437105 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:43.437119 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:43.437396 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:43.437643 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:43.437645 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:43.437647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:43.437770 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:43.438111 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:43.438115 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:43.438313 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 00:43:43.438457 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 00:43:43.438691 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":104,"threads_started":1}
I20260806 00:43:43.440527 30795 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38109
I20260806 00:43:43.440752 30795 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:43.441191 30795 heartbeater.cc:511] Master 127.29.84.190:38109 requested a full tablet report, sending...
I20260806 00:43:43.442137 30116 ts_manager.cc:194] Registered new tserver with Master: 72f38b0950864d66b9740c016f3f1383 (127.29.84.131:37053)
I20260806 00:43:43.442688 30116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:44393
I20260806 00:43:43.448791 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977023, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977023, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:43:43.474663 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:43.465547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47714 (local address 127.29.84.129:37723)
0806 00:43:43.465863 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:43.465868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:43.465891 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:43:43.466024 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:43.466030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:43.466205 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:43.466318 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:43.467144 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.467854 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.468643 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.468969 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.469049 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:43.471422 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:43.471431 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:43.471442 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:43.471465 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:43.472874 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:43.473320 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:43.473326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:43.473327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:43.473424 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:43.473734 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:43.473738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:43.473739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:43.473966 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:43.474181 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:43.474357 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 00:43:43.474507 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":104,"threads_started":1}
W20260806 00:43:43.475577 30277 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47714
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:43:43.500528 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:43.492706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.29.84.129:37723)
0806 00:43:43.492819 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:43.492823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:43.492980 (+ 157us) server_negotiation.cc:407] Connection header received
0806 00:43:43.493117 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:43.493120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:43.493160 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:43.493248 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:43.493864 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.494334 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.495176 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:43.495406 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:43.495517 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:43.497707 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:43.497711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:43.497713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:43.497742 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:43.498894 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:43.499340 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:43.499343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:43.499345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:43.499422 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:43.499691 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:43.499695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:43.499697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:43.499863 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:43.500006 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:43.500256 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 00:43:43.500368 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:43:43.501399 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30214
I20260806 00:43:43.513104 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30407
I20260806 00:43:43.531203 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30600
I20260806 00:43:43.541631 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30071
2026-08-06T00:43:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2711 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30819](info): set up 2 sockets
Aug 06 00:43:43 dist-test-slave-n326 krb5kdc[30819](info): commencing operation
krb5kdc: starting...
W20260806 00:43:44.756466 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:43:44 dist-test-slave-n326 krb5kdc[30819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:43:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:43:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:43:44.875511 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40857
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:40857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:44.978071 30842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:44.978267 30842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:44.978320 30842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:44.981989 30842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:43:44.982033 30842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:44.982052 30842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:44.982069 30842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:43:44.982084 30842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:43:44.986829 30842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:40857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:44.987706 30842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:44.988514 30842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:44.996377 30852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:44.996439 30842 server_base.cc:1034] running on GCE node
W20260806 00:43:44.996383 30848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:44.996562 30849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:44.997139 30842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:44.998347 30842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:44.999518 30842 hybrid_clock.cc:648] HybridClock initialized: now 1785977024999497 us; error 36 us; skew 500 ppm
Aug 06 00:43:44 dist-test-slave-n326 krb5kdc[30819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977024, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:45.002256 30842 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:43:45.003136 30842 webserver.cc:460] Webserver started at http://127.29.84.190:40733/ using document root <none> and password file <none>
I20260806 00:43:45.003695 30842 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:45.003752 30842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:45.004001 30842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:45.005826 30842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "48d9648357234bc38b8352f598d40ef9"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "fae14ab48819ca37acf4313a36780876"
server_key_iv: "b28b0e3c833b2f98c27799b790584269"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.006366 30842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "48d9648357234bc38b8352f598d40ef9"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "fae14ab48819ca37acf4313a36780876"
server_key_iv: "b28b0e3c833b2f98c27799b790584269"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.009733 30842 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:43:45.011967 30864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.013028 30842 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:43:45.013155 30842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "48d9648357234bc38b8352f598d40ef9"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "fae14ab48819ca37acf4313a36780876"
server_key_iv: "b28b0e3c833b2f98c27799b790584269"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.013252 30842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:45.027719 30842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:45.030124 30842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:45.030289 30842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:45.036911 30842 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:40857
I20260806 00:43:45.036927 30945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:40857 every 8 connection(s)
I20260806 00:43:45.037717 30842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:43:45.040524 30946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:45.040928 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30842
I20260806 00:43:45.041035 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:43:45.041273 30034 external_mini_cluster.cc:1428] Setting key d0cb609ea233e01d86de1b101c52225c
I20260806 00:43:45.047011 30946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9: Bootstrap starting.
I20260806 00:43:45.049602 30946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9: Neither blocks nor log segments found. Creating new log.
Aug 06 00:43:45 dist-test-slave-n326 krb5kdc[30819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:45.050374 30946 log.cc:824] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:45.052433 30946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9: No bootstrap required, opened a new log
I20260806 00:43:45.054751 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.042531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32944 (local address 127.29.84.190:40857)
0806 00:43:45.042946 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:45.042969 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:45.043001 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:43:45.043676 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:45.043691 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:45.043975 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:45.044223 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:45.045148 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.046066 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.047233 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.047671 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.047788 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.050380 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:45.050389 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:45.050406 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:45.050435 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:45.052161 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.052684 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.052689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.052705 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.052809 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.053101 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.053103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.053104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.053460 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:45.053599 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:45.053871 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 00:43:45.054128 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":153,"threads_started":1}
I20260806 00:43:45.056293 30946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d9648357234bc38b8352f598d40ef9" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40857 } }
I20260806 00:43:45.056515 30946 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:45.056581 30946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48d9648357234bc38b8352f598d40ef9, State: Initialized, Role: FOLLOWER
I20260806 00:43:45.057060 30946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d9648357234bc38b8352f598d40ef9" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40857 } }
I20260806 00:43:45.057207 30946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:43:45.057289 30946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:43:45.057380 30946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:45.058130 30946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d9648357234bc38b8352f598d40ef9" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40857 } }
I20260806 00:43:45.058434 30946 leader_election.cc:302] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48d9648357234bc38b8352f598d40ef9; no voters:
I20260806 00:43:45.058704 30946 leader_election.cc:288] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:43:45.058801 30952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:45.059020 30952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [term 1 LEADER]: Becoming Leader. State: Replica: 48d9648357234bc38b8352f598d40ef9, State: Running, Role: LEADER
I20260806 00:43:45.059331 30952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d9648357234bc38b8352f598d40ef9" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40857 } }
I20260806 00:43:45.059859 30946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:43:45.060967 30955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48d9648357234bc38b8352f598d40ef9. Latest consensus state: current_term: 1 leader_uuid: "48d9648357234bc38b8352f598d40ef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d9648357234bc38b8352f598d40ef9" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40857 } } }
I20260806 00:43:45.061002 30954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48d9648357234bc38b8352f598d40ef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d9648357234bc38b8352f598d40ef9" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40857 } } }
I20260806 00:43:45.061113 30955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [sys.catalog]: This master's current role is: LEADER
I20260806 00:43:45.061115 30954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9 [sys.catalog]: This master's current role is: LEADER
I20260806 00:43:45.061486 30966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:43:45.064430 30966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:43:45.069778 30966 catalog_manager.cc:1347] Generated new cluster ID: cf966881226c41208bb5ac8954dae9dd
I20260806 00:43:45.069841 30966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:43:45.086621 30966 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:43:45.087721 30966 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:43:45.104185 30966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48d9648357234bc38b8352f598d40ef9: Generated new TSK 0
I20260806 00:43:45.104847 30966 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:43:45.146242 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40857
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:45.275710 30984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:45.275933 30984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:45.276010 30984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:45.279755 30984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:45.279801 30984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:45.279902 30984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:43:45.284773 30984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40857
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:45.285701 30984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:45.286633 30984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:45.295398 30993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:45.295452 30994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:45.295596 30984 server_base.cc:1034] running on GCE node
W20260806 00:43:45.295454 30996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:45.296109 30984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:45.296695 30984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:45.297858 30984 hybrid_clock.cc:648] HybridClock initialized: now 1785977025297840 us; error 30 us; skew 500 ppm
Aug 06 00:43:45 dist-test-slave-n326 krb5kdc[30819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977025, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:45.300489 30984 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:43:45.301272 30984 webserver.cc:460] Webserver started at http://127.29.84.129:36179/ using document root <none> and password file <none>
I20260806 00:43:45.301826 30984 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:45.301927 30984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:45.302145 30984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:45.304033 30984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "99ec0930b7264716ab734c92abbd651b"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "dbd7d80a438b69613b90303438b8a6fc"
server_key_iv: "de1bf560d28f9afd54678f6ee8f6aad5"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.304564 30984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "99ec0930b7264716ab734c92abbd651b"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "dbd7d80a438b69613b90303438b8a6fc"
server_key_iv: "de1bf560d28f9afd54678f6ee8f6aad5"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.308040 30984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:43:45.310322 31006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.311618 30984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:43:45.311757 30984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "99ec0930b7264716ab734c92abbd651b"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "dbd7d80a438b69613b90303438b8a6fc"
server_key_iv: "de1bf560d28f9afd54678f6ee8f6aad5"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.311864 30984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:45.329015 30984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:45.331568 30984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:45.331755 30984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:45.332273 30984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:43:45.333019 30984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:45.333091 30984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.333155 30984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:45.333201 30984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.342088 30984 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:42773
I20260806 00:43:45.342121 31174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:42773 every 8 connection(s)
I20260806 00:43:45.342942 30984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
Aug 06 00:43:45 dist-test-slave-n326 krb5kdc[30819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977025, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:45.352636 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30984
I20260806 00:43:45.352746 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:43:45.353015 30034 external_mini_cluster.cc:1428] Setting key f1fdf22069a1434b11ba1a1e12928cd6
I20260806 00:43:45.356922 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.344756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:45793 (local address 127.29.84.190:40857)
0806 00:43:45.344904 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:45.344909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:45.345614 (+ 705us) server_negotiation.cc:407] Connection header received
0806 00:43:45.346983 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:45.346987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:45.347047 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:45.347200 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:45.348291 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.348831 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.349912 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.350147 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.350257 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.352812 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:45.352816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:45.352819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:45.352849 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:45.354532 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.355426 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.355430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.355432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.355519 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.355875 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.355880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.355882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.356062 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:45.356181 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:45.356709 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 00:43:45.356806 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:43:45.358009 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.345029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40857 (local address 127.29.84.129:45793)
0806 00:43:45.345460 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:45.345500 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:45.346777 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:45.347367 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:45.347373 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:45.347705 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:45.348138 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.348150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.348960 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.348964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.349782 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.349791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.350292 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.350297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.350471 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.351007 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:45.351022 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:45.352582 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:45.354660 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:45.354704 (+ 44us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:45.354721 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:45.355316 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:45.355633 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:45.355636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:45.355638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:45.355776 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:45.356180 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:45.356187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:45.356453 (+ 266us) client_negotiation.cc:769] Sending connection context
0806 00:43:45.356666 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 00:43:45.356993 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":123,"threads_started":1}
I20260806 00:43:45.359334 31175 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40857
I20260806 00:43:45.359603 31175 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:45.360170 31175 heartbeater.cc:511] Master 127.29.84.190:40857 requested a full tablet report, sending...
I20260806 00:43:45.361927 30891 ts_manager.cc:194] Registered new tserver with Master: 99ec0930b7264716ab734c92abbd651b (127.29.84.129:42773)
I20260806 00:43:45.363574 30891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:45793
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:43:45.382385 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40857
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:45.487857 31181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:45.488060 31181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:45.488099 31181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:45.491912 31181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:45.491966 31181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:45.492019 31181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:43:45.496909 31181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40857
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:45.497716 31181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:45.498535 31181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:45.507367 31190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:45.507557 31181 server_base.cc:1034] running on GCE node
W20260806 00:43:45.507366 31192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:45.507419 31186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:45.508039 31181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:45.508556 31181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:45.509713 31181 hybrid_clock.cc:648] HybridClock initialized: now 1785977025509684 us; error 43 us; skew 500 ppm
Aug 06 00:43:45 dist-test-slave-n326 krb5kdc[30819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977025, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:45.512377 31181 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:43:45.513183 31181 webserver.cc:460] Webserver started at http://127.29.84.130:34971/ using document root <none> and password file <none>
I20260806 00:43:45.513651 31181 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:45.513706 31181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:45.513847 31181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:45.515807 31181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "f9e0542a53714daeab6ede4cb9150990"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "c315eb1e979f6ec116def9fefa93542a"
server_key_iv: "853373b4087b60a5d8dade668006f2a4"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.516353 31181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "f9e0542a53714daeab6ede4cb9150990"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "c315eb1e979f6ec116def9fefa93542a"
server_key_iv: "853373b4087b60a5d8dade668006f2a4"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.520066 31181 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:43:45.522275 31204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.523432 31181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:45.523522 31181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "f9e0542a53714daeab6ede4cb9150990"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "c315eb1e979f6ec116def9fefa93542a"
server_key_iv: "853373b4087b60a5d8dade668006f2a4"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.523603 31181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:45.558283 31181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:45.561050 31181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:45.561196 31181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:45.561810 31181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:43:45.562561 31181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:45.562609 31181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.562673 31181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:45.562721 31181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.571682 31181 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:37489
I20260806 00:43:45.571723 31362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:37489 every 8 connection(s)
I20260806 00:43:45.572510 31181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:43:45.577908 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 31181
I20260806 00:43:45.577994 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:43:45.578250 30034 external_mini_cluster.cc:1428] Setting key e93fc134bdb544eb3cf4d3d4d0b97e00
Aug 06 00:43:45 dist-test-slave-n326 krb5kdc[30819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977025, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:45.587016 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.574365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:39955 (local address 127.29.84.190:40857)
0806 00:43:45.574517 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:45.574521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:45.575272 (+ 751us) server_negotiation.cc:407] Connection header received
0806 00:43:45.576579 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:45.576582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:45.576639 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:45.576746 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:45.577885 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.578533 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.580031 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.580238 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.580320 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.583135 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:45.583142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:45.583147 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:45.583181 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:45.584901 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.585389 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.585395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.585400 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.585494 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.585829 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.585836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.585838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.586109 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:45.586224 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:45.586679 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 00:43:45.586816 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:43:45.587641 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.574651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40857 (local address 127.29.84.130:39955)
0806 00:43:45.575123 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:45.575175 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:45.576378 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:45.576909 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:45.576915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:45.577253 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:45.577737 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.577750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.578685 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.578688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.579906 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.579918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.580392 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.580398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.580655 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.581261 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:45.581286 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:45.582942 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:45.585023 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:45.585028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:45.585038 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:45.585280 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:45.585605 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:45.585607 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:45.585608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:45.585731 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:45.586232 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:45.586237 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:45.586439 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 00:43:45.586584 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 00:43:45.586812 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":109,"threads_started":1}
I20260806 00:43:45.588774 31365 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40857
I20260806 00:43:45.589032 31365 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:43:45.589578 31365 heartbeater.cc:511] Master 127.29.84.190:40857 requested a full tablet report, sending...
I20260806 00:43:45.590673 30891 ts_manager.cc:194] Registered new tserver with Master: f9e0542a53714daeab6ede4cb9150990 (127.29.84.130:37489)
I20260806 00:43:45.591234 30891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:39955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:43:45.605893 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40857
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:45.713203 31371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:45.713500 31371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:45.713568 31371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:45.717710 31371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:45.717784 31371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:45.717885 31371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:43:45.722851 31371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40857
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:45.723775 31371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:45.724670 31371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:45.733647 31377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:45.733652 31378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:45.733727 31384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:45.733974 31371 server_base.cc:1034] running on GCE node
I20260806 00:43:45.734350 31371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:45.734929 31371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:45.736129 31371 hybrid_clock.cc:648] HybridClock initialized: now 1785977025736115 us; error 26 us; skew 500 ppm
Aug 06 00:43:45 dist-test-slave-n326 krb5kdc[30819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977025, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:45.738667 31371 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:43:45.739487 31371 webserver.cc:460] Webserver started at http://127.29.84.131:33617/ using document root <none> and password file <none>
I20260806 00:43:45.740000 31371 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:45.740088 31371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:45.740305 31371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:45.742141 31371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "c78d3c810ada4285a3653a33e5d005e7"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "a2c6f20c24a4af20ca62bed6dc9f95f1"
server_key_iv: "b2097637cae04d59654c3f4f9475c540"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.742677 31371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "c78d3c810ada4285a3653a33e5d005e7"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "a2c6f20c24a4af20ca62bed6dc9f95f1"
server_key_iv: "b2097637cae04d59654c3f4f9475c540"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.746124 31371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:43:45.748389 31399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.749502 31371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:43:45.749629 31371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "c78d3c810ada4285a3653a33e5d005e7"
format_stamp: "Formatted at 2026-08-06 00:43:45 on dist-test-slave-n326"
server_key: "a2c6f20c24a4af20ca62bed6dc9f95f1"
server_key_iv: "b2097637cae04d59654c3f4f9475c540"
server_key_version: "encryptionkey@0"
I20260806 00:43:45.749727 31371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:45.775507 31371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:45.778044 31371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:45.778230 31371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:45.778760 31371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:43:45.779516 31371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:45.779587 31371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.779650 31371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:45.779698 31371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:45.788995 31371 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:37903
I20260806 00:43:45.789040 31556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:37903 every 8 connection(s)
I20260806 00:43:45.789788 31371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:43:45.791831 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 31371
I20260806 00:43:45.791951 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:43:45.792287 30034 external_mini_cluster.cc:1428] Setting key 88ecd8260e8e850ae04894fcf6b5bfdb
Aug 06 00:43:45 dist-test-slave-n326 krb5kdc[30819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977025, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:45.802428 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.791538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:43405 (local address 127.29.84.190:40857)
0806 00:43:45.791663 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:45.791666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:45.792596 (+ 930us) server_negotiation.cc:407] Connection header received
0806 00:43:45.793699 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:45.793702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:45.793773 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:45.793848 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:45.795024 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.795593 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.796705 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.796933 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.797042 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.799279 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:45.799282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:45.799285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:45.799314 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:45.800485 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.800960 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.800964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.800966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.801044 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.801409 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.801412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.801414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.801586 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:45.801694 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:45.802142 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 00:43:45.802253 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:43:45.803148 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.791932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40857 (local address 127.29.84.131:43405)
0806 00:43:45.792443 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:45.792494 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:45.793449 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:45.794015 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:45.794021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:45.794396 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:45.794858 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.794871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.795725 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.795729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.796578 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.796588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.797076 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.797080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.797240 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.797697 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:45.797712 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:45.799125 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:45.800601 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:45.800606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:45.800618 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:45.800852 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:45.801135 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:45.801139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:45.801141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:45.801288 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:45.801693 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:45.801699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:45.801916 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 00:43:45.802099 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:43:45.802382 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":148,"threads_started":1}
I20260806 00:43:45.804136 31557 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40857
I20260806 00:43:45.804356 31557 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:45.804813 31557 heartbeater.cc:511] Master 127.29.84.190:40857 requested a full tablet report, sending...
I20260806 00:43:45.805936 30890 ts_manager.cc:194] Registered new tserver with Master: c78d3c810ada4285a3653a33e5d005e7 (127.29.84.131:37903)
I20260806 00:43:45.806520 30890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:43405
I20260806 00:43:45.817617 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:43:45.827572 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.820758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.29.84.190:40857)
0806 00:43:45.820894 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:45.820897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:45.821070 (+ 173us) server_negotiation.cc:407] Connection header received
0806 00:43:45.821156 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:45.821158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:45.821192 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:45.821248 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:45.821883 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.822343 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.823210 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.823460 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.823538 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.824661 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:45.824664 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:45.824666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:45.824686 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:45.826075 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.826484 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.826486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.826487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.826559 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:45.826757 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:45.826759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:45.826760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:45.826927 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:45.827018 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:45.827292 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 00:43:45.827474 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 00:43:45.831655 30890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:43:45.834303 30890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:43:45.848074 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.842125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:42773 (local address 127.0.0.1:38670)
0806 00:43:45.842939 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:45.842953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:45.843047 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:45.843657 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:45.843660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:45.843678 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:45.843996 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.844005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.845114 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.845117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.846410 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.846421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.847541 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.847544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.847679 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.847717 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 00:43:45.847821 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:43:45.847921 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":162,"thread_start_us":105,"threads_started":1}
I20260806 00:43:45.848325 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.842066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49266 (local address 127.29.84.131:37903)
0806 00:43:45.842953 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:45.842966 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:45.842988 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:43:45.843055 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:45.843062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:45.843302 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:45.843448 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:45.844066 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.845261 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.846725 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.847545 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.847655 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.847782 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:45.848007 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 00:43:45.848111 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":148,"threads_started":1}
I20260806 00:43:45.848641 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.842478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.29.84.129:42773)
0806 00:43:45.842956 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:45.842975 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:45.843017 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:43:45.843172 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:45.843178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:45.843359 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:45.843503 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:45.844125 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.844966 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.846544 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.847360 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.847492 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.847619 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:45.848413 (+ 794us) server_negotiation.cc:299] Negotiation successful
0806 00:43:45.848511 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":146,"threads_started":1}
I20260806 00:43:45.848824 31586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.843026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:37489 (local address 127.0.0.1:57342)
0806 00:43:45.843869 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:45.843882 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:45.843969 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:45.844434 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:45.844437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:45.844455 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:45.844534 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.844539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.845893 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.845900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.847050 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.847062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.848402 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.848411 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.848510 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.848533 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:43:45.848630 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:43:45.848695 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":106,"threads_started":1}
I20260806 00:43:45.848996 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.843364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57342 (local address 127.29.84.130:37489)
0806 00:43:45.843814 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:45.843825 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:45.843963 (+ 138us) server_negotiation.cc:407] Connection header received
0806 00:43:45.844098 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:45.844105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:45.844278 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:45.844420 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:45.844657 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.845762 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.847196 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.848043 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.848138 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.848273 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:45.848664 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 00:43:45.848833 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":125,"threads_started":1}
I20260806 00:43:45.849201 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:45.841976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:37903 (local address 127.0.0.1:49266)
0806 00:43:45.842481 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:45.842529 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:45.842680 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:45.843629 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:45.843635 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:45.843659 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:45.843815 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.843823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.845398 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.845401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.846603 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:45.846613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:45.847695 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:45.847698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:45.847849 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:45.847878 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 00:43:45.847959 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 00:43:45.848032 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":3,"thread_start_us":72,"threads_started":1}
I20260806 00:43:45.851649 31077 tablet_service.cc:1467] Processing CreateTablet for tablet 9caced8cc7844414a1ba66418419f3fd (DEFAULT_TABLE table=test-table [id=f93664d1bb1840de9a8a86e12e87c57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:45.852268 31266 tablet_service.cc:1467] Processing CreateTablet for tablet 9caced8cc7844414a1ba66418419f3fd (DEFAULT_TABLE table=test-table [id=f93664d1bb1840de9a8a86e12e87c57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:45.852771 31077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9caced8cc7844414a1ba66418419f3fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:45.852373 31466 tablet_service.cc:1467] Processing CreateTablet for tablet 9caced8cc7844414a1ba66418419f3fd (DEFAULT_TABLE table=test-table [id=f93664d1bb1840de9a8a86e12e87c57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:45.853230 31266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9caced8cc7844414a1ba66418419f3fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:45.853376 31466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9caced8cc7844414a1ba66418419f3fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:45.860116 31589 tablet_bootstrap.cc:492] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990: Bootstrap starting.
I20260806 00:43:45.862653 31592 tablet_bootstrap.cc:492] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7: Bootstrap starting.
I20260806 00:43:45.863118 31589 tablet_bootstrap.cc:654] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:45.863137 31590 tablet_bootstrap.cc:492] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b: Bootstrap starting.
I20260806 00:43:45.864414 31589 log.cc:824] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:45.865870 31592 tablet_bootstrap.cc:654] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:45.866323 31589 tablet_bootstrap.cc:492] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990: No bootstrap required, opened a new log
I20260806 00:43:45.866307 31590 tablet_bootstrap.cc:654] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:45.866541 31589 ts_tablet_manager.cc:1397] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:43:45.867835 31592 log.cc:824] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:45.867836 31590 log.cc:824] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:45.869830 31590 tablet_bootstrap.cc:492] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b: No bootstrap required, opened a new log
I20260806 00:43:45.869874 31592 tablet_bootstrap.cc:492] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7: No bootstrap required, opened a new log
I20260806 00:43:45.870010 31590 ts_tablet_manager.cc:1397] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260806 00:43:45.870009 31592 ts_tablet_manager.cc:1397] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 00:43:45.871495 31589 raft_consensus.cc:348] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } }
I20260806 00:43:45.871778 31589 raft_consensus.cc:374] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:45.871836 31589 raft_consensus.cc:729] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e0542a53714daeab6ede4cb9150990, State: Initialized, Role: FOLLOWER
I20260806 00:43:45.872274 31589 consensus_queue.cc:260] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } }
I20260806 00:43:45.872951 31365 heartbeater.cc:503] Master 127.29.84.190:40857 was elected leader, sending a full tablet report...
I20260806 00:43:45.873230 31589 ts_tablet_manager.cc:1428] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:43:45.876223 31592 raft_consensus.cc:348] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } }
I20260806 00:43:45.876557 31592 raft_consensus.cc:374] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:45.876415 31590 raft_consensus.cc:348] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } }
I20260806 00:43:45.876627 31592 raft_consensus.cc:729] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78d3c810ada4285a3653a33e5d005e7, State: Initialized, Role: FOLLOWER
I20260806 00:43:45.876654 31590 raft_consensus.cc:374] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:45.876719 31590 raft_consensus.cc:729] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ec0930b7264716ab734c92abbd651b, State: Initialized, Role: FOLLOWER
I20260806 00:43:45.877048 31590 consensus_queue.cc:260] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } }
I20260806 00:43:45.877048 31592 consensus_queue.cc:260] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } }
I20260806 00:43:45.877745 31557 heartbeater.cc:503] Master 127.29.84.190:40857 was elected leader, sending a full tablet report...
I20260806 00:43:45.878038 31175 heartbeater.cc:503] Master 127.29.84.190:40857 was elected leader, sending a full tablet report...
I20260806 00:43:45.878078 31592 ts_tablet_manager.cc:1428] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260806 00:43:45.878507 31590 ts_tablet_manager.cc:1428] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b: Time spent starting tablet: real 0.008s user 0.005s sys 0.005s
I20260806 00:43:45.999629 31601 raft_consensus.cc:482] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:43:45.999883 31601 raft_consensus.cc:504] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } }
I20260806 00:43:46.001691 31601 leader_election.cc:288] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99ec0930b7264716ab734c92abbd651b (127.29.84.129:42773), f9e0542a53714daeab6ede4cb9150990 (127.29.84.130:37489)
I20260806 00:43:46.006481 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:46.002634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:45153 (local address 127.29.84.129:42773)
0806 00:43:46.002782 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:46.002788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:46.002802 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:43:46.002846 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:46.002849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:46.002893 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:46.002969 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:46.003513 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.004071 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.005210 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.005846 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.005939 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:46.005979 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:46.006285 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 00:43:46.006342 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:43:46.006488 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:46.001960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:42773 (local address 127.29.84.131:45153)
0806 00:43:46.002454 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:46.002471 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:46.002600 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:46.003012 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:46.003016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:46.003042 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:46.003337 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:46.003353 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.004192 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.004195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:46.005070 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:46.005080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.005965 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.005969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:46.006149 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:46.006176 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 00:43:46.006263 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 00:43:46.006338 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":1}
I20260806 00:43:46.007166 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:46.002296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:45891 (local address 127.29.84.130:37489)
0806 00:43:46.002430 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:46.002434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:46.002638 (+ 204us) server_negotiation.cc:407] Connection header received
0806 00:43:46.002825 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:46.002829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:46.002881 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:46.002947 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:46.003607 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.004071 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.005758 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.006386 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.006500 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:46.006565 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:46.006955 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 00:43:46.007031 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:43:46.007211 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:46.001976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:37489 (local address 127.29.84.131:45891)
0806 00:43:46.002505 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:46.002520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:46.002704 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:46.003149 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:46.003154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:46.003180 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:46.003492 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:46.003502 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.004199 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.004202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:46.005627 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:46.005639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.006600 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.006606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:46.006752 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:46.006795 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 00:43:46.006909 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 00:43:46.006997 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":35,"thread_start_us":127,"threads_started":1}
I20260806 00:43:46.007261 31105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caced8cc7844414a1ba66418419f3fd" candidate_uuid: "c78d3c810ada4285a3653a33e5d005e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ec0930b7264716ab734c92abbd651b" is_pre_election: true
I20260806 00:43:46.007542 31105 raft_consensus.cc:2454] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78d3c810ada4285a3653a33e5d005e7 in term 0.
I20260806 00:43:46.008049 31405 leader_election.cc:302] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99ec0930b7264716ab734c92abbd651b, c78d3c810ada4285a3653a33e5d005e7; no voters:
I20260806 00:43:46.008379 31601 raft_consensus.cc:2781] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:43:46.008371 31298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caced8cc7844414a1ba66418419f3fd" candidate_uuid: "c78d3c810ada4285a3653a33e5d005e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e0542a53714daeab6ede4cb9150990" is_pre_election: true
I20260806 00:43:46.008479 31601 raft_consensus.cc:482] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:43:46.008540 31601 raft_consensus.cc:3037] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:46.008683 31298 raft_consensus.cc:2454] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78d3c810ada4285a3653a33e5d005e7 in term 0.
I20260806 00:43:46.010145 31601 raft_consensus.cc:504] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } }
I20260806 00:43:46.010514 31601 leader_election.cc:288] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [CANDIDATE]: Term 1 election: Requested vote from peers 99ec0930b7264716ab734c92abbd651b (127.29.84.129:42773), f9e0542a53714daeab6ede4cb9150990 (127.29.84.130:37489)
I20260806 00:43:46.010905 31105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caced8cc7844414a1ba66418419f3fd" candidate_uuid: "c78d3c810ada4285a3653a33e5d005e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ec0930b7264716ab734c92abbd651b"
I20260806 00:43:46.010990 31298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caced8cc7844414a1ba66418419f3fd" candidate_uuid: "c78d3c810ada4285a3653a33e5d005e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e0542a53714daeab6ede4cb9150990"
I20260806 00:43:46.011041 31105 raft_consensus.cc:3037] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:46.011152 31298 raft_consensus.cc:3037] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:46.012678 31298 raft_consensus.cc:2454] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78d3c810ada4285a3653a33e5d005e7 in term 1.
I20260806 00:43:46.012874 31105 raft_consensus.cc:2454] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78d3c810ada4285a3653a33e5d005e7 in term 1.
I20260806 00:43:46.013054 31407 leader_election.cc:302] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c78d3c810ada4285a3653a33e5d005e7, f9e0542a53714daeab6ede4cb9150990; no voters:
I20260806 00:43:46.013345 31601 raft_consensus.cc:2781] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:46.013579 31601 raft_consensus.cc:686] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [term 1 LEADER]: Becoming Leader. State: Replica: c78d3c810ada4285a3653a33e5d005e7, State: Running, Role: LEADER
I20260806 00:43:46.013953 31601 consensus_queue.cc:237] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } }
I20260806 00:43:46.017222 30891 catalog_manager.cc:5495] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78d3c810ada4285a3653a33e5d005e7 (127.29.84.131). New cstate: current_term: 1 leader_uuid: "c78d3c810ada4285a3653a33e5d005e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78d3c810ada4285a3653a33e5d005e7" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 37903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99ec0930b7264716ab734c92abbd651b" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 42773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e0542a53714daeab6ede4cb9150990" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 37489 } health_report { overall_health: UNKNOWN } } }
W20260806 00:43:46.040699 31559 tablet.cc:2369] T 9caced8cc7844414a1ba66418419f3fd P c78d3c810ada4285a3653a33e5d005e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 00:43:46 dist-test-slave-n326 krb5kdc[30819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:43:46.048242 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:46.033030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.29.84.129:42773)
0806 00:43:46.033148 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:46.033153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:46.033470 (+ 317us) server_negotiation.cc:407] Connection header received
0806 00:43:46.033627 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:46.033630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:46.033679 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:46.033756 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:46.034575 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.035158 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.036320 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.036542 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.036646 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:46.044535 (+ 7889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:46.044545 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:46.044557 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:46.044580 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:46.046465 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:46.046951 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:46.046956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:46.046957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:46.047036 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:46.047342 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:46.047344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:46.047345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:46.047585 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:46.047728 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:46.047954 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 00:43:46.048049 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 00:43:46 dist-test-slave-n326 krb5kdc[30819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:43:46 dist-test-slave-n326 krb5kdc[30819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
W20260806 00:43:46.073585 31366 tablet.cc:2369] T 9caced8cc7844414a1ba66418419f3fd P f9e0542a53714daeab6ede4cb9150990: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:43:46.076833 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:46.068447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.29.84.129:42773)
0806 00:43:46.068580 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:46.068584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:46.068738 (+ 154us) server_negotiation.cc:407] Connection header received
0806 00:43:46.068890 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:46.068897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:46.068937 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:46.069021 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:46.069758 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.070346 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.071206 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.071479 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.071593 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:46.073949 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:46.073952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:46.073954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:46.073980 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:46.075202 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:46.075615 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:46.075619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:46.075621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:46.075698 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:46.075969 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:46.075972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:46.075981 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:46.076135 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:46.076270 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:46.076565 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 00:43:46.076681 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 00:43:46 dist-test-slave-n326 krb5kdc[30819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260806 00:43:46.094308 31176 tablet.cc:2369] T 9caced8cc7844414a1ba66418419f3fd P 99ec0930b7264716ab734c92abbd651b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 00:43:46 dist-test-slave-n326 krb5kdc[30819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:43:46.102388 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:46.094679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.29.84.129:42773)
0806 00:43:46.094791 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:46.094796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:46.094960 (+ 164us) server_negotiation.cc:407] Connection header received
0806 00:43:46.095128 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:46.095131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:46.095178 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:46.095260 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:46.095876 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.096348 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.097183 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:46.097388 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:46.097497 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:46.099774 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:46.099778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:46.099779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:46.099800 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:46.100976 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:46.101348 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:46.101351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:46.101352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:46.101439 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:46.101674 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:46.101676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:46.101677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:46.101818 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:46.101922 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:46.102099 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 00:43:46.102184 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 00:43:46.103765 31050 tablet_service.cc:2908] Rejecting scan request for tablet 9caced8cc7844414a1ba66418419f3fd: Uninitialized: clean time has not yet been initialized
I20260806 00:43:46.105677 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30984
I20260806 00:43:46.116382 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 31181
I20260806 00:43:46.133751 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 31371
I20260806 00:43:46.143388 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30842
2026-08-06T00:43:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2601 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:43:46 dist-test-slave-n326 krb5kdc[31623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:43:46 dist-test-slave-n326 krb5kdc[31623](info): set up 2 sockets
Aug 06 00:43:46 dist-test-slave-n326 krb5kdc[31623](info): commencing operation
krb5kdc: starting...
W20260806 00:43:47.347580 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:43:47 dist-test-slave-n326 krb5kdc[31623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:43:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:43:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:43:47.466100 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40969
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:40969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:47.570942 31644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:47.571208 31644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:47.571280 31644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:47.574976 31644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:43:47.575050 31644 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 00:43:47.575078 31644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:47.575140 31644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:47.575199 31644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:43:47.575248 31644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:43:47.580075 31644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:40969
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:47.580961 31644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:47.581887 31644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:47.589587 31653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:47.589588 31651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:47.589593 31656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:47.589924 31644 server_base.cc:1034] running on GCE node
I20260806 00:43:47.590449 31644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:47.591482 31644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:47.592662 31644 hybrid_clock.cc:648] HybridClock initialized: now 1785977027592634 us; error 41 us; skew 500 ppm
Aug 06 00:43:47 dist-test-slave-n326 krb5kdc[31623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977027, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:47.595353 31644 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:43:47.596130 31644 webserver.cc:460] Webserver started at http://127.29.84.190:36087/ using document root <none> and password file <none>
I20260806 00:43:47.596658 31644 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:47.596747 31644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:47.596966 31644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:47.598788 31644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "e0755cfade724faca23a015c539cdf0e"
format_stamp: "Formatted at 2026-08-06 00:43:47 on dist-test-slave-n326"
server_key: "0d20cdb0a98f82becceb1d0f2f4cc8c2"
server_key_iv: "f4d4efe48914f3a8be57c0cd6545aa58"
server_key_version: "encryptionkey@0"
I20260806 00:43:47.599350 31644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "e0755cfade724faca23a015c539cdf0e"
format_stamp: "Formatted at 2026-08-06 00:43:47 on dist-test-slave-n326"
server_key: "0d20cdb0a98f82becceb1d0f2f4cc8c2"
server_key_iv: "f4d4efe48914f3a8be57c0cd6545aa58"
server_key_version: "encryptionkey@0"
I20260806 00:43:47.602849 31644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:43:47.605034 31665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:47.606114 31644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:47.606251 31644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "e0755cfade724faca23a015c539cdf0e"
format_stamp: "Formatted at 2026-08-06 00:43:47 on dist-test-slave-n326"
server_key: "0d20cdb0a98f82becceb1d0f2f4cc8c2"
server_key_iv: "f4d4efe48914f3a8be57c0cd6545aa58"
server_key_version: "encryptionkey@0"
I20260806 00:43:47.606357 31644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:47.630506 31644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:47.633049 31644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:47.633235 31644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:47.640683 31644 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:40969
I20260806 00:43:47.640709 31760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:40969 every 8 connection(s)
I20260806 00:43:47.641531 31644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:43:47.642007 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 31644
I20260806 00:43:47.642146 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:43:47.642405 30034 external_mini_cluster.cc:1428] Setting key 270ae79a83a5a894e6c137250566e2e8
I20260806 00:43:47.644466 31762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:47.650727 31762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e: Bootstrap starting.
Aug 06 00:43:47 dist-test-slave-n326 krb5kdc[31623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:47.653203 31762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:47.654399 31762 log.cc:824] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:47.656384 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:47.643639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44478 (local address 127.29.84.190:40969)
0806 00:43:47.644022 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:47.644039 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:47.644166 (+ 127us) server_negotiation.cc:407] Connection header received
0806 00:43:47.644961 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:47.644982 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:47.645367 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:47.645698 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:47.646333 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.647823 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.648655 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.649312 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.649461 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:47.651974 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:47.651979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:47.651990 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:47.652014 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:47.653592 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:47.654073 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:47.654077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:47.654081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:47.654191 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:47.654466 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:47.654468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:47.654469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:47.654919 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:47.655206 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:47.655442 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 00:43:47.655726 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":107,"threads_started":1}
I20260806 00:43:47.656389 31762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e: No bootstrap required, opened a new log
I20260806 00:43:47.659641 31762 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0755cfade724faca23a015c539cdf0e" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40969 } }
I20260806 00:43:47.659907 31762 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:47.659973 31762 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0755cfade724faca23a015c539cdf0e, State: Initialized, Role: FOLLOWER
I20260806 00:43:47.660392 31762 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0755cfade724faca23a015c539cdf0e" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40969 } }
I20260806 00:43:47.660513 31762 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:43:47.660595 31762 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:43:47.660698 31762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:47.661506 31762 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0755cfade724faca23a015c539cdf0e" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40969 } }
I20260806 00:43:47.661818 31762 leader_election.cc:302] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0755cfade724faca23a015c539cdf0e; no voters:
I20260806 00:43:47.662088 31762 leader_election.cc:288] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:43:47.662218 31773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:47.662441 31773 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [term 1 LEADER]: Becoming Leader. State: Replica: e0755cfade724faca23a015c539cdf0e, State: Running, Role: LEADER
I20260806 00:43:47.662727 31773 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0755cfade724faca23a015c539cdf0e" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40969 } }
I20260806 00:43:47.663252 31762 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:43:47.664294 31775 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0755cfade724faca23a015c539cdf0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0755cfade724faca23a015c539cdf0e" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40969 } } }
I20260806 00:43:47.664422 31775 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [sys.catalog]: This master's current role is: LEADER
I20260806 00:43:47.664708 31776 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0755cfade724faca23a015c539cdf0e. Latest consensus state: current_term: 1 leader_uuid: "e0755cfade724faca23a015c539cdf0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0755cfade724faca23a015c539cdf0e" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40969 } } }
I20260806 00:43:47.664811 31776 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e [sys.catalog]: This master's current role is: LEADER
I20260806 00:43:47.664780 31784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:43:47.667851 31784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:43:47.672957 31784 catalog_manager.cc:1347] Generated new cluster ID: f6a985d937624575ab40e7abf6dce0f5
I20260806 00:43:47.673022 31784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:43:47.685248 31784 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:43:47.686311 31784 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:43:47.695811 31784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e: Generated new TSK 0
I20260806 00:43:47.696466 31784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:43:47.730197 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40969
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:47.839552 31802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:47.839767 31802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:47.839807 31802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:47.844036 31802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:47.844089 31802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:47.844129 31802 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:43:47.844197 31802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:43:47.849179 31802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.84.190:40969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:47.850037 31802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:47.850893 31802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:47.859449 31815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:47.859472 31812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:47.859551 31802 server_base.cc:1034] running on GCE node
W20260806 00:43:47.859474 31811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:47.860144 31802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:47.860721 31802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:47.861912 31802 hybrid_clock.cc:648] HybridClock initialized: now 1785977027861904 us; error 27 us; skew 500 ppm
Aug 06 00:43:47 dist-test-slave-n326 krb5kdc[31623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977027, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:47.864759 31802 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:43:47.865588 31802 webserver.cc:460] Webserver started at http://127.29.84.129:39847/ using document root <none> and password file <none>
I20260806 00:43:47.866115 31802 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:47.866206 31802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:47.866434 31802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:47.868364 31802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "03b390384dec44d3b266c9842d093fc0"
format_stamp: "Formatted at 2026-08-06 00:43:47 on dist-test-slave-n326"
server_key: "3dffdbcc18d5d0646a3803e543c23869"
server_key_iv: "94a13d2629466494b7260f9949e66c4f"
server_key_version: "encryptionkey@0"
I20260806 00:43:47.868908 31802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "03b390384dec44d3b266c9842d093fc0"
format_stamp: "Formatted at 2026-08-06 00:43:47 on dist-test-slave-n326"
server_key: "3dffdbcc18d5d0646a3803e543c23869"
server_key_iv: "94a13d2629466494b7260f9949e66c4f"
server_key_version: "encryptionkey@0"
I20260806 00:43:47.872277 31802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:43:47.874347 31825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:47.875555 31802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:43:47.875686 31802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "03b390384dec44d3b266c9842d093fc0"
format_stamp: "Formatted at 2026-08-06 00:43:47 on dist-test-slave-n326"
server_key: "3dffdbcc18d5d0646a3803e543c23869"
server_key_iv: "94a13d2629466494b7260f9949e66c4f"
server_key_version: "encryptionkey@0"
I20260806 00:43:47.875790 31802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:47.895673 31802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:47.898268 31802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:47.898555 31802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:47.900019 31802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:47.900098 31802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:47.900161 31802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:47.900213 31802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:43:47 dist-test-slave-n326 krb5kdc[31623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977027, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:47.910844 31802 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:45625
I20260806 00:43:47.911201 32009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:45625 every 8 connection(s)
I20260806 00:43:47.911763 31802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:43:47.914045 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:47.900656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:56449 (local address 127.29.84.190:40969)
0806 00:43:47.900820 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:47.900825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:47.902078 (+ 1253us) server_negotiation.cc:407] Connection header received
0806 00:43:47.903265 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:47.903268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:47.903328 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:47.903432 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:47.904985 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.905903 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.907492 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.907795 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.907953 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:47.910525 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:47.910531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:47.910537 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:47.910571 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:47.912130 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:47.912840 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:47.912843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:47.912844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:47.912922 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:47.913149 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:47.913150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:47.913151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:47.913296 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:47.913382 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:47.913823 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 00:43:47.913933 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:43:47.914772 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:47.901478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40969 (local address 127.29.84.129:56449)
0806 00:43:47.901915 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:47.901963 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:47.902993 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:47.903616 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:47.903624 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:47.904121 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:47.904799 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:47.904821 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.906065 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.906068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:47.907334 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:47.907348 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.907988 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.907992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:47.908154 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:47.908707 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:47.908730 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:47.910354 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:47.912419 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:47.912424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:47.912437 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:47.912739 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:47.912984 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:47.912986 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:47.912987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:47.913095 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:47.913426 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:47.913430 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:47.913624 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 00:43:47.913770 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 00:43:47.913996 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":129,"threads_started":1}
I20260806 00:43:47.915805 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 31802
I20260806 00:43:47.915923 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:43:47.916133 32011 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40969
I20260806 00:43:47.916221 30034 external_mini_cluster.cc:1428] Setting key 17d5f1e632fffa4e401229cf69e81243
I20260806 00:43:47.916311 32011 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:47.916863 32011 heartbeater.cc:511] Master 127.29.84.190:40969 requested a full tablet report, sending...
I20260806 00:43:47.918534 31686 ts_manager.cc:194] Registered new tserver with Master: 03b390384dec44d3b266c9842d093fc0 (127.29.84.129:45625)
I20260806 00:43:47.919694 31686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:56449
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:43:47.926584 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:47.920018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:49601 (local address 127.29.84.190:40969)
0806 00:43:47.920145 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:47.920150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:47.920166 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:43:47.920206 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:47.920211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:47.920269 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:47.920355 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:47.920835 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.921463 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.922333 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.922568 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.922676 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:47.923673 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:47.923676 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:47.923678 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:47.923706 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:47.925022 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:47.925467 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:47.925470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:47.925472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:47.925549 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:47.925828 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:47.925833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:47.925834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:47.925996 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:47.926100 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:47.926352 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 00:43:47.926468 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:43:47.926595 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:47.918187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40969 (local address 127.29.84.129:49601)
0806 00:43:47.918463 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:47.918477 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:47.918560 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:47.920392 (+ 1832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:47.920395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:47.920600 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:47.920710 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:47.920717 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.921598 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.921602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:47.922206 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:47.922215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:47.922680 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:47.922683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:47.922771 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:47.923263 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:47.923282 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:47.923547 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:47.925152 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:47.925155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:47.925157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:47.925365 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:47.925644 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:47.925648 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:47.925650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:47.925736 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:47.926093 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:47.926096 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:47.926190 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 00:43:47.926298 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 00:43:47.926436 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:43:47.945222 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40969
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:48.052404 32022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:48.052619 32022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:48.052659 32022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:48.056535 32022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:48.056577 32022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:48.056612 32022 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:43:48.056668 32022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:43:48.061305 32022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.84.190:40969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:48.062155 32022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:48.063128 32022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:48.071945 32030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:48.072041 32033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:48.072050 32029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:48.072367 32022 server_base.cc:1034] running on GCE node
I20260806 00:43:48.072695 32022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:48.073247 32022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:48.074416 32022 hybrid_clock.cc:648] HybridClock initialized: now 1785977028074389 us; error 41 us; skew 500 ppm
Aug 06 00:43:48 dist-test-slave-n326 krb5kdc[31623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:48.077064 32022 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:43:48.077868 32022 webserver.cc:460] Webserver started at http://127.29.84.130:37879/ using document root <none> and password file <none>
I20260806 00:43:48.078413 32022 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:48.078509 32022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:48.078711 32022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:48.080588 32022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "175d3741a54b4588adc20c4f1fc915ce"
format_stamp: "Formatted at 2026-08-06 00:43:48 on dist-test-slave-n326"
server_key: "66a744c786dabc99c2a03f895ac02758"
server_key_iv: "e2726c3b86883bfdcf7b7ddb093b959d"
server_key_version: "encryptionkey@0"
I20260806 00:43:48.081046 32022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "175d3741a54b4588adc20c4f1fc915ce"
format_stamp: "Formatted at 2026-08-06 00:43:48 on dist-test-slave-n326"
server_key: "66a744c786dabc99c2a03f895ac02758"
server_key_iv: "e2726c3b86883bfdcf7b7ddb093b959d"
server_key_version: "encryptionkey@0"
I20260806 00:43:48.084365 32022 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:43:48.086570 32042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:48.087776 32022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:48.087867 32022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "175d3741a54b4588adc20c4f1fc915ce"
format_stamp: "Formatted at 2026-08-06 00:43:48 on dist-test-slave-n326"
server_key: "66a744c786dabc99c2a03f895ac02758"
server_key_iv: "e2726c3b86883bfdcf7b7ddb093b959d"
server_key_version: "encryptionkey@0"
I20260806 00:43:48.087988 32022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:48.111805 32022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:48.114194 32022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:48.114369 32022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:48.115703 32022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:48.115759 32022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:48.115824 32022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:48.115864 32022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:43:48 dist-test-slave-n326 krb5kdc[31623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:48.126190 32022 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:41435
I20260806 00:43:48.126253 32223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:41435 every 8 connection(s)
I20260806 00:43:48.126977 32022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:43:48.128976 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.116458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:52965 (local address 127.29.84.190:40969)
0806 00:43:48.116600 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.116605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.117796 (+ 1191us) server_negotiation.cc:407] Connection header received
0806 00:43:48.118751 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.118754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.118809 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.118914 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:48.119986 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.120554 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.121729 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.121981 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.122116 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.125515 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:48.125518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:48.125520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:48.125544 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:48.127189 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.127769 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.127773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.127774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.127867 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.128154 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.128156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.128157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.128305 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:48.128374 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.128769 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.128856 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:43:48.129730 32059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.117266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40969 (local address 127.29.84.130:52965)
0806 00:43:48.117653 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.117696 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.118559 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.119099 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.119110 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.119451 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:48.119842 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.119855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.120678 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.120681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.121602 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.121612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.122163 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.122167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.122369 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.122967 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:48.122984 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:48.125353 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:48.127301 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.127309 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.127325 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.127664 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.127952 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.127954 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.127956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.128076 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.128384 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:48.128388 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:48.128565 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.128707 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.128925 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":112,"threads_started":1}
I20260806 00:43:48.130695 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 32022
I20260806 00:43:48.130811 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:43:48.130923 32224 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40969
I20260806 00:43:48.131057 32224 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:48.131227 30034 external_mini_cluster.cc:1428] Setting key 4c8d6eedacf096b3e88a15a370ea0d72
I20260806 00:43:48.131618 32224 heartbeater.cc:511] Master 127.29.84.190:40969 requested a full tablet report, sending...
I20260806 00:43:48.132680 31687 ts_manager.cc:194] Registered new tserver with Master: 175d3741a54b4588adc20c4f1fc915ce (127.29.84.130:41435)
I20260806 00:43:48.133409 31687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:52965
I20260806 00:43:48.140585 32235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.133135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40969 (local address 127.29.84.130:40735)
0806 00:43:48.133396 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.133409 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.133504 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.133947 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.133950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.134162 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:48.134297 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.134306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.135361 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.135363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.135874 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.135881 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.136487 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.136489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.136553 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.136981 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:48.136992 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:48.137278 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:48.139211 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.139213 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.139215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.139395 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.139695 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.139697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.139698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.139767 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.140234 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:48.140236 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:48.140302 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.140377 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.140468 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":2,"thread_start_us":122,"threads_started":1}
I20260806 00:43:48.140738 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.133187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:40735 (local address 127.29.84.190:40969)
0806 00:43:48.133728 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.133734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.133749 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:43:48.133784 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.133789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.133832 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.133921 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:48.134428 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.135239 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.135989 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.136360 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.136492 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.137425 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:48.137432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:48.137438 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:48.137470 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:48.139106 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.139495 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.139500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.139504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.139594 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.139853 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.139856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.139859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.140131 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:48.140242 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.140459 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.140578 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465}
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:43:48.159958 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40969
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:48.268014 32239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:48.268224 32239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:48.268301 32239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:48.272225 32239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:48.272277 32239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:48.272313 32239 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:43:48.272343 32239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:43:48.277285 32239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.84.190:40969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:48.278087 32239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:48.279115 32239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:48.288084 32255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:48.288103 32248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:48.288333 32244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:48.288379 32239 server_base.cc:1034] running on GCE node
I20260806 00:43:48.288753 32239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:48.289299 32239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:48.290501 32239 hybrid_clock.cc:648] HybridClock initialized: now 1785977028290480 us; error 53 us; skew 500 ppm
Aug 06 00:43:48 dist-test-slave-n326 krb5kdc[31623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:48.293320 32239 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:43:48.294080 32239 webserver.cc:460] Webserver started at http://127.29.84.131:46563/ using document root <none> and password file <none>
I20260806 00:43:48.294613 32239 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:48.294713 32239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:48.294957 32239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:48.296963 32239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "060a77fb83504edfbe7ddb7c56bebca9"
format_stamp: "Formatted at 2026-08-06 00:43:48 on dist-test-slave-n326"
server_key: "e353e326625452ebb7d401f74c44df36"
server_key_iv: "f46ec556d4397e3f6356df6b36636383"
server_key_version: "encryptionkey@0"
I20260806 00:43:48.297493 32239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "060a77fb83504edfbe7ddb7c56bebca9"
format_stamp: "Formatted at 2026-08-06 00:43:48 on dist-test-slave-n326"
server_key: "e353e326625452ebb7d401f74c44df36"
server_key_iv: "f46ec556d4397e3f6356df6b36636383"
server_key_version: "encryptionkey@0"
I20260806 00:43:48.300889 32239 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:43:48.303128 32263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:48.304255 32239 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:43:48.304504 32239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "060a77fb83504edfbe7ddb7c56bebca9"
format_stamp: "Formatted at 2026-08-06 00:43:48 on dist-test-slave-n326"
server_key: "e353e326625452ebb7d401f74c44df36"
server_key_iv: "f46ec556d4397e3f6356df6b36636383"
server_key_version: "encryptionkey@0"
I20260806 00:43:48.304615 32239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:48.320308 32239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:48.322827 32239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:48.323031 32239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:48.324311 32239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:48.324383 32239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:48.324474 32239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:48.324527 32239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:43:48 dist-test-slave-n326 krb5kdc[31623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:48.335546 32239 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:39597
I20260806 00:43:48.335954 32430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:39597 every 8 connection(s)
I20260806 00:43:48.336477 32239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:43:48.338762 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.325138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:37215 (local address 127.29.84.190:40969)
0806 00:43:48.325288 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.325292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.326214 (+ 922us) server_negotiation.cc:407] Connection header received
0806 00:43:48.327255 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.327259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.327329 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.327394 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:48.329204 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.330195 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.331963 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.332242 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.332350 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.334905 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:48.334910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:48.334913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:48.334941 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:48.336675 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.337276 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.337280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.337282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.337371 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.337717 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.337721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.337723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.337905 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:48.338013 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.338468 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.338585 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:43:48.339497 32274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.325623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40969 (local address 127.29.84.131:37215)
0806 00:43:48.326072 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.326113 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.326963 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.327907 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.327918 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.328396 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:48.329032 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.329050 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.330361 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.330364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.331825 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.331837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.332416 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.332420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.332595 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.333130 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:48.333156 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:48.334732 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:48.336791 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.336796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.336808 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.337160 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.337466 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.337470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.337472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.337615 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.338011 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:48.338018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:48.338243 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.338423 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.338709 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":131,"threads_started":1}
I20260806 00:43:48.340584 32431 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40969
I20260806 00:43:48.340740 32431 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:48.341336 32431 heartbeater.cc:511] Master 127.29.84.190:40969 requested a full tablet report, sending...
I20260806 00:43:48.342368 31686 ts_manager.cc:194] Registered new tserver with Master: 060a77fb83504edfbe7ddb7c56bebca9 (127.29.84.131:39597)
I20260806 00:43:48.343135 31686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:37215
I20260806 00:43:48.345499 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 32239
I20260806 00:43:48.345623 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:43:48.345890 30034 external_mini_cluster.cc:1428] Setting key c979c90c487e78c19dfe2bdd666ef51c
I20260806 00:43:48.347942 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:43:48.350929 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.342734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:37899 (local address 127.29.84.190:40969)
0806 00:43:48.342851 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.342855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.342993 (+ 138us) server_negotiation.cc:407] Connection header received
0806 00:43:48.343104 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.343107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.343150 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.343230 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:48.344757 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.345208 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.346167 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.346462 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.346579 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.347683 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:48.347687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:48.347689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:48.347714 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:48.349203 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.349638 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.349644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.349648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.349740 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.350025 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.350031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.350036 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.350272 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:48.350378 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.350730 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.350833 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:43:48.351255 32441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.342640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40969 (local address 127.29.84.131:37899)
0806 00:43:48.342918 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.342929 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.343003 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.343272 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.343275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.343492 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:48.343608 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.343613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.345375 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.345377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.346057 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.346066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.346595 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.346600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.346717 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.347244 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:48.347260 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:48.347564 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:48.349339 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.349341 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.349343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.349544 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.349844 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.349846 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.349847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.349917 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.350417 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:48.350421 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:48.350539 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.350656 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.350844 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":94,"threads_started":1}
Aug 06 00:43:48 dist-test-slave-n326 krb5kdc[31623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977028, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:43:48 dist-test-slave-n326 krb5kdc[31623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977028, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:48.373936 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.366441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.29.84.190:40969)
0806 00:43:48.366567 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.366570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.366580 (+ 10us) server_negotiation.cc:407] Connection header received
0806 00:43:48.366622 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.366624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.366653 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.366735 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:48.367439 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.367906 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.368706 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.368904 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.369006 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.371181 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:48.371187 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:48.371194 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:48.371216 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:48.372372 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.372791 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.372794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.372795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.372862 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.373136 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.373139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.373141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.373292 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:48.373427 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.373683 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.373788 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:43:48.376463 31686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:43:48.379462 31686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:43:48.392524 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.386838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:45625 (local address 127.0.0.1:51442)
0806 00:43:48.387215 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.387261 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.387369 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.388357 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.388362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.388380 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:48.388520 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.388526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.389697 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.389702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.390821 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.390832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.392004 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.392008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.392114 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.392153 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.392262 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.392372 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"spinlock_wait_cycles":896,"thread_start_us":94,"threads_started":1}
I20260806 00:43:48.392926 32458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.387855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.29.84.131:39597)
0806 00:43:48.388201 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.388214 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.388436 (+ 222us) server_negotiation.cc:407] Connection header received
0806 00:43:48.388511 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.388516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.388688 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.388850 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:48.389050 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.390130 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.391538 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.392349 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.392454 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.392604 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.392703 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.392790 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":100,"threads_started":1}
I20260806 00:43:48.392935 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.387043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.29.84.129:45625)
0806 00:43:48.387736 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.387753 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.387775 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:43:48.387849 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.387856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.388052 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.388201 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:48.388643 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.389593 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.390963 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.391862 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.391984 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.392134 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.392724 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.392805 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":73,"threads_started":1}
I20260806 00:43:48.392871 32455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.387514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:39597 (local address 127.0.0.1:35414)
0806 00:43:48.388328 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.388340 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.388427 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.388827 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.388829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.388846 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:48.388945 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.388952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.390258 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.390261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.391415 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.391426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.392477 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.392484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.392579 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.392600 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.392673 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.392743 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":44,"thread_start_us":79,"threads_started":1}
I20260806 00:43:48.393419 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.387292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:41435 (local address 127.0.0.1:49390)
0806 00:43:48.388075 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.388091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.388172 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.389035 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.389038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.389055 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:48.389146 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.389151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.390428 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.390431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.391474 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.391503 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.392769 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.392773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.392863 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.392889 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.392964 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.393056 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":91,"threads_started":1}
I20260806 00:43:48.393951 32457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.387712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.29.84.130:41435)
0806 00:43:48.388437 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.388450 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.388474 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:43:48.388545 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.388548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.388735 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.388898 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:48.389269 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.390300 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.391617 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.392426 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.392532 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.393647 (+ 1115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.393700 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.393776 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":113,"threads_started":1}
I20260806 00:43:48.396054 31908 tablet_service.cc:1467] Processing CreateTablet for tablet c1c0e005acea4c6d9f4bd4bf490b15d8 (DEFAULT_TABLE table=test-table [id=c86b8c45509043db8e4b3f6ecb839fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:48.396474 32329 tablet_service.cc:1467] Processing CreateTablet for tablet c1c0e005acea4c6d9f4bd4bf490b15d8 (DEFAULT_TABLE table=test-table [id=c86b8c45509043db8e4b3f6ecb839fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:48.397233 31908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c0e005acea4c6d9f4bd4bf490b15d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:48.397394 32329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c0e005acea4c6d9f4bd4bf490b15d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:48.397362 32128 tablet_service.cc:1467] Processing CreateTablet for tablet c1c0e005acea4c6d9f4bd4bf490b15d8 (DEFAULT_TABLE table=test-table [id=c86b8c45509043db8e4b3f6ecb839fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:48.398238 32128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c0e005acea4c6d9f4bd4bf490b15d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:48.404587 32459 tablet_bootstrap.cc:492] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: Bootstrap starting.
I20260806 00:43:48.405620 32461 tablet_bootstrap.cc:492] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: Bootstrap starting.
I20260806 00:43:48.406900 32459 tablet_bootstrap.cc:654] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:48.407346 32461 tablet_bootstrap.cc:654] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:48.407392 32460 tablet_bootstrap.cc:492] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: Bootstrap starting.
I20260806 00:43:48.407821 32459 log.cc:824] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:48.408162 32461 log.cc:824] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:48.409327 32460 tablet_bootstrap.cc:654] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:48.409971 32460 log.cc:824] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:48.410058 32459 tablet_bootstrap.cc:492] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: No bootstrap required, opened a new log
I20260806 00:43:48.410287 32459 ts_tablet_manager.cc:1397] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:43:48.412114 32460 tablet_bootstrap.cc:492] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: No bootstrap required, opened a new log
I20260806 00:43:48.412295 32460 ts_tablet_manager.cc:1397] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 00:43:48.413259 32461 tablet_bootstrap.cc:492] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: No bootstrap required, opened a new log
I20260806 00:43:48.413475 32461 ts_tablet_manager.cc:1397] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 00:43:48.414106 32459 raft_consensus.cc:348] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } }
I20260806 00:43:48.414330 32459 raft_consensus.cc:374] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:48.414383 32459 raft_consensus.cc:729] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175d3741a54b4588adc20c4f1fc915ce, State: Initialized, Role: FOLLOWER
I20260806 00:43:48.414788 32459 consensus_queue.cc:260] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } }
I20260806 00:43:48.415431 32224 heartbeater.cc:503] Master 127.29.84.190:40969 was elected leader, sending a full tablet report...
I20260806 00:43:48.415992 32460 raft_consensus.cc:348] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } }
I20260806 00:43:48.416273 32460 raft_consensus.cc:374] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:48.416329 32460 raft_consensus.cc:729] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 060a77fb83504edfbe7ddb7c56bebca9, State: Initialized, Role: FOLLOWER
I20260806 00:43:48.416810 32459 ts_tablet_manager.cc:1428] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 00:43:48.416749 32460 consensus_queue.cc:260] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } }
I20260806 00:43:48.417456 32431 heartbeater.cc:503] Master 127.29.84.190:40969 was elected leader, sending a full tablet report...
I20260806 00:43:48.417315 32461 raft_consensus.cc:348] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } }
I20260806 00:43:48.417559 32461 raft_consensus.cc:374] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:48.417606 32461 raft_consensus.cc:729] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b390384dec44d3b266c9842d093fc0, State: Initialized, Role: FOLLOWER
I20260806 00:43:48.417690 32460 ts_tablet_manager.cc:1428] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:43:48.417949 32461 consensus_queue.cc:260] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } }
I20260806 00:43:48.418514 32011 heartbeater.cc:503] Master 127.29.84.190:40969 was elected leader, sending a full tablet report...
I20260806 00:43:48.419652 32461 ts_tablet_manager.cc:1428] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:43:48.484634 32467 raft_consensus.cc:482] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:43:48.484900 32467 raft_consensus.cc:504] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } }
I20260806 00:43:48.486045 32467 leader_election.cc:288] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03b390384dec44d3b266c9842d093fc0 (127.29.84.129:45625), 060a77fb83504edfbe7ddb7c56bebca9 (127.29.84.131:39597)
I20260806 00:43:48.489687 32059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.486287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:45625 (local address 127.29.84.130:40339)
0806 00:43:48.486416 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.486432 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.486534 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.486860 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.486862 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.486895 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:48.487001 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.487005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.487801 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.487806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.488572 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.488581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.489212 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.489216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.489382 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.489400 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.489486 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.489559 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":25}
I20260806 00:43:48.489687 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.486345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:40339 (local address 127.29.84.129:45625)
0806 00:43:48.486478 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.486494 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.486508 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:43:48.486657 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.486661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.486712 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.486802 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:48.487150 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.487685 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.488701 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.489093 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.489223 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.489275 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.489503 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.489572 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:43:48.490512 32458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.486740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:35813 (local address 127.29.84.131:39597)
0806 00:43:48.486956 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.486962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.487007 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:43:48.487196 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.487199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.487245 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.487328 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:48.487969 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.488458 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.489490 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.489993 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.490128 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.490173 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.490342 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.490409 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260806 00:43:48.490525 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.486579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:39597 (local address 127.29.84.130:35813)
0806 00:43:48.486937 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.486950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.487046 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.487340 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.487343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.487358 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:48.487849 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.487857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.488568 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.488575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.489370 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.489379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.490128 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.490132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.490219 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.490239 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.490314 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.490372 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":37,"thread_start_us":165,"threads_started":1}
I20260806 00:43:48.490484 31945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0e005acea4c6d9f4bd4bf490b15d8" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b390384dec44d3b266c9842d093fc0" is_pre_election: true
I20260806 00:43:48.490773 31945 raft_consensus.cc:2454] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 0.
I20260806 00:43:48.491276 32048 leader_election.cc:302] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b390384dec44d3b266c9842d093fc0, 175d3741a54b4588adc20c4f1fc915ce; no voters:
I20260806 00:43:48.491266 32351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0e005acea4c6d9f4bd4bf490b15d8" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060a77fb83504edfbe7ddb7c56bebca9" is_pre_election: true
I20260806 00:43:48.491546 32351 raft_consensus.cc:2454] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 0.
I20260806 00:43:48.491564 32467 raft_consensus.cc:2781] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:43:48.491639 32467 raft_consensus.cc:482] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:43:48.491691 32467 raft_consensus.cc:3037] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:48.493232 32467 raft_consensus.cc:504] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } }
I20260806 00:43:48.493561 32467 leader_election.cc:288] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 election: Requested vote from peers 03b390384dec44d3b266c9842d093fc0 (127.29.84.129:45625), 060a77fb83504edfbe7ddb7c56bebca9 (127.29.84.131:39597)
I20260806 00:43:48.493953 31945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0e005acea4c6d9f4bd4bf490b15d8" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b390384dec44d3b266c9842d093fc0"
I20260806 00:43:48.493986 32351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0e005acea4c6d9f4bd4bf490b15d8" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060a77fb83504edfbe7ddb7c56bebca9"
I20260806 00:43:48.494072 31945 raft_consensus.cc:3037] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:48.494112 32351 raft_consensus.cc:3037] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:48.495247 31945 raft_consensus.cc:2454] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 1.
I20260806 00:43:48.495564 32048 leader_election.cc:302] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b390384dec44d3b266c9842d093fc0, 175d3741a54b4588adc20c4f1fc915ce; no voters:
I20260806 00:43:48.495771 32467 raft_consensus.cc:2781] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:48.495775 32351 raft_consensus.cc:2454] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 1.
I20260806 00:43:48.495970 32467 raft_consensus.cc:686] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 1 LEADER]: Becoming Leader. State: Replica: 175d3741a54b4588adc20c4f1fc915ce, State: Running, Role: LEADER
I20260806 00:43:48.496289 32467 consensus_queue.cc:237] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } }
I20260806 00:43:48.499310 31686 catalog_manager.cc:5495] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 175d3741a54b4588adc20c4f1fc915ce (127.29.84.130). New cstate: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } health_report { overall_health: HEALTHY } } }
I20260806 00:43:48.520663 32457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.516732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49402 (local address 127.29.84.130:41435)
0806 00:43:48.516860 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.516864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.516958 (+ 94us) server_negotiation.cc:407] Connection header received
0806 00:43:48.517190 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.517194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.517270 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.517386 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:43:48.517753 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.518215 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.519166 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.519381 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.519492 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.519892 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:43:48.520248 (+ 356us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:43:48.520373 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.520490 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.520541 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:43:48.526271 31945 raft_consensus.cc:1261] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Refusing update from remote peer 175d3741a54b4588adc20c4f1fc915ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:48.526818 32351 raft_consensus.cc:1261] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Refusing update from remote peer 175d3741a54b4588adc20c4f1fc915ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:48.527087 32472 consensus_queue.cc:1035] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:48.527459 32467 consensus_queue.cc:1035] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:48.534346 32480 mvcc.cc:205] Tried to move back new op lower bound from 7315361908836544512 to 7315361908723335168. Current Snapshot: MvccSnapshot[applied={T|T < 7315361908836544512}]
I20260806 00:43:48.547511 31686 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:48.547641 31686 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:48.550043 31686 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e: Sending DeleteTablet for 3 replicas of tablet c1c0e005acea4c6d9f4bd4bf490b15d8
I20260806 00:43:48.550894 32128 tablet_service.cc:1514] Processing DeleteTablet for tablet c1c0e005acea4c6d9f4bd4bf490b15d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:48 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:49390
I20260806 00:43:48.550876 32329 tablet_service.cc:1514] Processing DeleteTablet for tablet c1c0e005acea4c6d9f4bd4bf490b15d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:48 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:35414
I20260806 00:43:48.551546 32490 tablet_replica.cc:330] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: stopping tablet replica
I20260806 00:43:48.551577 32491 tablet_replica.cc:330] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: stopping tablet replica
I20260806 00:43:48.552002 31908 tablet_service.cc:1514] Processing DeleteTablet for tablet c1c0e005acea4c6d9f4bd4bf490b15d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:48 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:51442
I20260806 00:43:48.552059 32490 raft_consensus.cc:2229] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:43:48.552547 32490 raft_consensus.cc:2258] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:48.552582 32491 raft_consensus.cc:2229] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:48.552587 31686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 00:43:48.552855 32491 raft_consensus.cc:2258] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 00:43:48.553000 31686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:43:48.554279 32491 ts_tablet_manager.cc:1905] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:48.554279 32490 ts_tablet_manager.cc:1905] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:48.555222 32492 tablet_replica.cc:330] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: stopping tablet replica
I20260806 00:43:48.555557 32492 raft_consensus.cc:2229] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:48.555984 32492 raft_consensus.cc:2258] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:48.557824 32491 ts_tablet_manager.cc:1918] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:43:48.557938 32491 log.cc:1196] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/c1c0e005acea4c6d9f4bd4bf490b15d8
I20260806 00:43:48.558317 32491 ts_tablet_manager.cc:1939] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 060a77fb83504edfbe7ddb7c56bebca9: Deleting consensus metadata
I20260806 00:43:48.559620 31908 tablet_service.cc:1467] Processing CreateTablet for tablet efdc876ef60c461c924a94e69a21e56a (DEFAULT_TABLE table=test-table [id=36413f8de8324d2e94f4fed8802d0c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:48.559684 32492 ts_tablet_manager.cc:1905] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:48.559815 32329 tablet_service.cc:1467] Processing CreateTablet for tablet efdc876ef60c461c924a94e69a21e56a (DEFAULT_TABLE table=test-table [id=36413f8de8324d2e94f4fed8802d0c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:48.560039 32329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdc876ef60c461c924a94e69a21e56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:48.560068 32128 tablet_service.cc:1467] Processing CreateTablet for tablet efdc876ef60c461c924a94e69a21e56a (DEFAULT_TABLE table=test-table [id=36413f8de8324d2e94f4fed8802d0c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:48.560289 32128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdc876ef60c461c924a94e69a21e56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:48.562462 32460 tablet_bootstrap.cc:492] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: Bootstrap starting.
I20260806 00:43:48.563179 31908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdc876ef60c461c924a94e69a21e56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:48.563545 32460 tablet_bootstrap.cc:654] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:48.564541 32492 ts_tablet_manager.cc:1918] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:43:48.564643 32492 log.cc:1196] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/c1c0e005acea4c6d9f4bd4bf490b15d8
I20260806 00:43:48.565013 32492 ts_tablet_manager.cc:1939] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 03b390384dec44d3b266c9842d093fc0: Deleting consensus metadata
I20260806 00:43:48.565843 32461 tablet_bootstrap.cc:492] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: Bootstrap starting.
I20260806 00:43:48.566246 32459 tablet_bootstrap.cc:492] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: Bootstrap starting.
I20260806 00:43:48.566777 32461 tablet_bootstrap.cc:654] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:48.566968 31669 catalog_manager.cc:4841] TS 060a77fb83504edfbe7ddb7c56bebca9 (127.29.84.131:39597): tablet c1c0e005acea4c6d9f4bd4bf490b15d8 (table test-table [id=c86b8c45509043db8e4b3f6ecb839fbe]) successfully deleted
I20260806 00:43:48.567200 31669 catalog_manager.cc:4841] TS 03b390384dec44d3b266c9842d093fc0 (127.29.84.129:45625): tablet c1c0e005acea4c6d9f4bd4bf490b15d8 (table test-table [id=c86b8c45509043db8e4b3f6ecb839fbe]) successfully deleted
I20260806 00:43:48.567731 32459 tablet_bootstrap.cc:654] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:48.567999 32490 ts_tablet_manager.cc:1918] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:43:48.568100 32490 log.cc:1196] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/c1c0e005acea4c6d9f4bd4bf490b15d8
I20260806 00:43:48.568169 32461 tablet_bootstrap.cc:492] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: No bootstrap required, opened a new log
I20260806 00:43:48.568279 32461 ts_tablet_manager.cc:1397] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:48.568466 32490 ts_tablet_manager.cc:1939] T c1c0e005acea4c6d9f4bd4bf490b15d8 P 175d3741a54b4588adc20c4f1fc915ce: Deleting consensus metadata
I20260806 00:43:48.569089 32461 raft_consensus.cc:348] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.569206 32461 raft_consensus.cc:374] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:48.569247 32461 raft_consensus.cc:729] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b390384dec44d3b266c9842d093fc0, State: Initialized, Role: FOLLOWER
I20260806 00:43:48.569386 32461 consensus_queue.cc:260] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.569531 31668 catalog_manager.cc:4841] TS 175d3741a54b4588adc20c4f1fc915ce (127.29.84.130:41435): tablet c1c0e005acea4c6d9f4bd4bf490b15d8 (table test-table [id=c86b8c45509043db8e4b3f6ecb839fbe]) successfully deleted
I20260806 00:43:48.570142 32460 tablet_bootstrap.cc:492] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: No bootstrap required, opened a new log
I20260806 00:43:48.570235 32460 ts_tablet_manager.cc:1397] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260806 00:43:48.570297 32461 ts_tablet_manager.cc:1428] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:43:48.570528 32459 tablet_bootstrap.cc:492] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: No bootstrap required, opened a new log
I20260806 00:43:48.570614 32459 ts_tablet_manager.cc:1397] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 00:43:48.570672 32460 raft_consensus.cc:348] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.570792 32460 raft_consensus.cc:374] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:48.570835 32460 raft_consensus.cc:729] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 060a77fb83504edfbe7ddb7c56bebca9, State: Initialized, Role: FOLLOWER
I20260806 00:43:48.570950 32460 consensus_queue.cc:260] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.571213 32460 ts_tablet_manager.cc:1428] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:43:48.571285 32459 raft_consensus.cc:348] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.571403 32459 raft_consensus.cc:374] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:48.571444 32459 raft_consensus.cc:729] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175d3741a54b4588adc20c4f1fc915ce, State: Initialized, Role: FOLLOWER
I20260806 00:43:48.571571 32459 consensus_queue.cc:260] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.571847 32459 ts_tablet_manager.cc:1428] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260806 00:43:48.587383 32432 tablet.cc:2369] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:43:48.628044 32225 tablet.cc:2369] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:43:48.663168 32014 tablet.cc:2369] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:43:48.712119 32472 raft_consensus.cc:482] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:43:48.712231 32472 raft_consensus.cc:504] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.712718 32472 leader_election.cc:288] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03b390384dec44d3b266c9842d093fc0 (127.29.84.129:45625), 060a77fb83504edfbe7ddb7c56bebca9 (127.29.84.131:39597)
I20260806 00:43:48.713137 31945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdc876ef60c461c924a94e69a21e56a" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b390384dec44d3b266c9842d093fc0" is_pre_election: true
I20260806 00:43:48.713299 31945 raft_consensus.cc:2454] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 0.
I20260806 00:43:48.713215 32351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdc876ef60c461c924a94e69a21e56a" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060a77fb83504edfbe7ddb7c56bebca9" is_pre_election: true
I20260806 00:43:48.713440 32351 raft_consensus.cc:2454] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 0.
I20260806 00:43:48.713672 32048 leader_election.cc:302] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b390384dec44d3b266c9842d093fc0, 175d3741a54b4588adc20c4f1fc915ce; no voters:
I20260806 00:43:48.713838 32472 raft_consensus.cc:2781] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:43:48.713896 32472 raft_consensus.cc:482] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:43:48.713935 32472 raft_consensus.cc:3037] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:48.715332 32472 raft_consensus.cc:504] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.715723 32472 leader_election.cc:288] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 election: Requested vote from peers 03b390384dec44d3b266c9842d093fc0 (127.29.84.129:45625), 060a77fb83504edfbe7ddb7c56bebca9 (127.29.84.131:39597)
I20260806 00:43:48.716071 32351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdc876ef60c461c924a94e69a21e56a" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060a77fb83504edfbe7ddb7c56bebca9"
I20260806 00:43:48.716192 32351 raft_consensus.cc:3037] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:48.716228 31945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdc876ef60c461c924a94e69a21e56a" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b390384dec44d3b266c9842d093fc0"
I20260806 00:43:48.716346 31945 raft_consensus.cc:3037] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:48.717634 31945 raft_consensus.cc:2454] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 1.
I20260806 00:43:48.717948 32048 leader_election.cc:302] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b390384dec44d3b266c9842d093fc0, 175d3741a54b4588adc20c4f1fc915ce; no voters:
I20260806 00:43:48.718010 32351 raft_consensus.cc:2454] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 1.
I20260806 00:43:48.718176 32472 raft_consensus.cc:2781] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:48.718277 32472 raft_consensus.cc:686] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 1 LEADER]: Becoming Leader. State: Replica: 175d3741a54b4588adc20c4f1fc915ce, State: Running, Role: LEADER
I20260806 00:43:48.718415 32472 consensus_queue.cc:237] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.720005 31686 catalog_manager.cc:5495] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 175d3741a54b4588adc20c4f1fc915ce (127.29.84.130). New cstate: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } health_report { overall_health: UNKNOWN } } }
Aug 06 00:43:48 dist-test-slave-n326 krb5kdc[31623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977027, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:48.773181 32510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.764906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40969 (local address 127.0.0.1:44496)
0806 00:43:48.765393 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:48.765405 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:48.765500 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:48.765708 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:48.765710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:48.766119 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:48.766264 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.766271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.766948 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.766950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.767551 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:48.767560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.767949 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.767951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:48.768067 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.768696 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:48.768709 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:48.770187 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:48.771891 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.771899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.771903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.772119 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.772353 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:48.772355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:48.772356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:48.772430 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:48.772743 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:48.772748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:48.772839 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 00:43:48.772927 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 00:43:48.773025 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":190,"threads_started":1}
I20260806 00:43:48.773192 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:48.764951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.29.84.190:40969)
0806 00:43:48.765060 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:48.765063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:48.765482 (+ 419us) server_negotiation.cc:407] Connection header received
0806 00:43:48.765565 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:48.765567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:48.765613 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:48.765691 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:48.766368 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.766862 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.767657 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:48.767852 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:48.767927 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:48.770326 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:48.770329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:48.770331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:48.770352 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:48.771797 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.772202 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.772204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.772206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.772276 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:48.772504 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:48.772506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:48.772507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:48.772656 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:48.772729 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:48.772985 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 00:43:48.773081 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 00:43:48.777135 31686 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:44496:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 00:43:48.777805 31686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:43:48.782547 31908 tablet_service.cc:1467] Processing CreateTablet for tablet 06b1082b179b4d56a185fedf107a3d45 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d93f1adcc8a435aaafcbc83b6d8f5f7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:43:48.782665 32128 tablet_service.cc:1467] Processing CreateTablet for tablet 06b1082b179b4d56a185fedf107a3d45 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d93f1adcc8a435aaafcbc83b6d8f5f7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:43:48.782765 32329 tablet_service.cc:1467] Processing CreateTablet for tablet 06b1082b179b4d56a185fedf107a3d45 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d93f1adcc8a435aaafcbc83b6d8f5f7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:43:48.782970 31908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b1082b179b4d56a185fedf107a3d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:48.783104 32128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b1082b179b4d56a185fedf107a3d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:48.783150 32329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b1082b179b4d56a185fedf107a3d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:48.785619 32459 tablet_bootstrap.cc:492] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: Bootstrap starting.
I20260806 00:43:48.786185 32461 tablet_bootstrap.cc:492] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: Bootstrap starting.
I20260806 00:43:48.786541 32460 tablet_bootstrap.cc:492] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: Bootstrap starting.
I20260806 00:43:48.786624 32459 tablet_bootstrap.cc:654] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:48.787979 32461 tablet_bootstrap.cc:654] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:48.788115 32460 tablet_bootstrap.cc:654] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:48.789711 32460 tablet_bootstrap.cc:492] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: No bootstrap required, opened a new log
I20260806 00:43:48.789798 32460 ts_tablet_manager.cc:1397] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 00:43:48.789866 32459 tablet_bootstrap.cc:492] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: No bootstrap required, opened a new log
I20260806 00:43:48.789976 32459 ts_tablet_manager.cc:1397] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 00:43:48.790498 32459 raft_consensus.cc:348] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.790490 32460 raft_consensus.cc:348] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.790617 32459 raft_consensus.cc:374] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:48.790676 32460 raft_consensus.cc:374] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:48.790669 32459 raft_consensus.cc:729] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175d3741a54b4588adc20c4f1fc915ce, State: Initialized, Role: FOLLOWER
I20260806 00:43:48.790719 32460 raft_consensus.cc:729] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 060a77fb83504edfbe7ddb7c56bebca9, State: Initialized, Role: FOLLOWER
I20260806 00:43:48.790838 32459 consensus_queue.cc:260] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.790838 32460 consensus_queue.cc:260] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.791108 32459 ts_tablet_manager.cc:1428] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:43:48.791118 32460 ts_tablet_manager.cc:1428] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 00:43:48.791340 32468 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:48.791565 32468 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:48.791778 32472 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:48.791917 32472 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:48.792732 32461 tablet_bootstrap.cc:492] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: No bootstrap required, opened a new log
I20260806 00:43:48.792824 32461 ts_tablet_manager.cc:1397] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260806 00:43:48.793569 32461 raft_consensus.cc:348] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.793692 32461 raft_consensus.cc:374] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:48.793740 32461 raft_consensus.cc:729] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b390384dec44d3b266c9842d093fc0, State: Initialized, Role: FOLLOWER
I20260806 00:43:48.793861 32461 consensus_queue.cc:260] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.794128 32461 ts_tablet_manager.cc:1428] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:43:48.794198 32469 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:48.794334 32469 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:48.973896 32472 raft_consensus.cc:482] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:43:48.974043 32472 raft_consensus.cc:504] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.974416 32472 leader_election.cc:288] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03b390384dec44d3b266c9842d093fc0 (127.29.84.129:45625), 060a77fb83504edfbe7ddb7c56bebca9 (127.29.84.131:39597)
I20260806 00:43:48.974861 31945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b1082b179b4d56a185fedf107a3d45" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b390384dec44d3b266c9842d093fc0" is_pre_election: true
I20260806 00:43:48.974912 32351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b1082b179b4d56a185fedf107a3d45" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060a77fb83504edfbe7ddb7c56bebca9" is_pre_election: true
I20260806 00:43:48.975019 31945 raft_consensus.cc:2454] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 0.
I20260806 00:43:48.975142 32351 raft_consensus.cc:2454] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 0.
I20260806 00:43:48.975404 32048 leader_election.cc:302] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b390384dec44d3b266c9842d093fc0, 175d3741a54b4588adc20c4f1fc915ce; no voters:
I20260806 00:43:48.975608 32472 raft_consensus.cc:2781] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:43:48.975664 32472 raft_consensus.cc:482] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:43:48.975692 32472 raft_consensus.cc:3037] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:48.976855 32472 raft_consensus.cc:504] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.977180 32472 leader_election.cc:288] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 election: Requested vote from peers 03b390384dec44d3b266c9842d093fc0 (127.29.84.129:45625), 060a77fb83504edfbe7ddb7c56bebca9 (127.29.84.131:39597)
I20260806 00:43:48.977571 31945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b1082b179b4d56a185fedf107a3d45" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b390384dec44d3b266c9842d093fc0"
I20260806 00:43:48.977660 31945 raft_consensus.cc:3037] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:48.977680 32351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b1082b179b4d56a185fedf107a3d45" candidate_uuid: "175d3741a54b4588adc20c4f1fc915ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060a77fb83504edfbe7ddb7c56bebca9"
I20260806 00:43:48.977799 32351 raft_consensus.cc:3037] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:48.978615 31945 raft_consensus.cc:2454] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 1.
I20260806 00:43:48.978906 32048 leader_election.cc:302] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b390384dec44d3b266c9842d093fc0, 175d3741a54b4588adc20c4f1fc915ce; no voters:
I20260806 00:43:48.978956 32351 raft_consensus.cc:2454] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175d3741a54b4588adc20c4f1fc915ce in term 1.
I20260806 00:43:48.979192 32472 raft_consensus.cc:2781] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:48.979293 32472 raft_consensus.cc:686] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [term 1 LEADER]: Becoming Leader. State: Replica: 175d3741a54b4588adc20c4f1fc915ce, State: Running, Role: LEADER
I20260806 00:43:48.979435 32472 consensus_queue.cc:237] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } }
I20260806 00:43:48.980085 32467 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: TxnStatusTablet state changed. Reason: New leader 175d3741a54b4588adc20c4f1fc915ce. Latest consensus state: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:48.980212 32467 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:43:48.980474 32518 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 00:43:48.980679 32518 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 00:43:48.981084 31687 catalog_manager.cc:5495] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 175d3741a54b4588adc20c4f1fc915ce (127.29.84.130). New cstate: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260806 00:43:49.025394 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:49.021643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.29.84.130:41435)
0806 00:43:49.021838 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:49.021842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:49.021858 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:43:49.022091 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:49.022093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:49.022139 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:49.022220 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:43:49.022570 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.023144 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.024368 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.024609 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.024684 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:49.024993 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:43:49.025057 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:43:49.025110 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:49.025240 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 00:43:49.025281 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260806 00:43:49.025434 32510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:49.021556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:41435 (local address 127.0.0.1:49418)
0806 00:43:49.021808 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:49.021824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:49.021930 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:49.022296 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:49.022300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:49.022310 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:43:49.022445 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:49.022453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.023245 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.023248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:49.024243 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:49.024254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.024711 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.024714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:49.024802 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:49.024855 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:43:49.025137 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:43:49.025151 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 00:43:49.025219 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 00:43:49.025299 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260806 00:43:49.027882 32351 raft_consensus.cc:1261] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Refusing update from remote peer 175d3741a54b4588adc20c4f1fc915ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:49.027969 31945 raft_consensus.cc:1261] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Refusing update from remote peer 175d3741a54b4588adc20c4f1fc915ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:49.028337 32467 consensus_queue.cc:1035] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:49.028467 32472 consensus_queue.cc:1035] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:49.029769 32468 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: TxnStatusTablet state changed. Reason: New leader 175d3741a54b4588adc20c4f1fc915ce. Latest consensus state: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:49.029876 32468 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:49.030162 32469 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: TxnStatusTablet state changed. Reason: New leader 175d3741a54b4588adc20c4f1fc915ce. Latest consensus state: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:49.030305 32469 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:49.032506 32472 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:49.032603 32472 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:43:49.032675 32477 mvcc.cc:205] Tried to move back new op lower bound from 7315361910893826048 to 7315361910700900352. Current Snapshot: MvccSnapshot[applied={T|T < 7315361910893826048}]
I20260806 00:43:49.032861 32468 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:49.032959 32468 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 060a77fb83504edfbe7ddb7c56bebca9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:49.036612 32472 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:49.036720 32472 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 175d3741a54b4588adc20c4f1fc915ce: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:43:49.037748 32479 mvcc.cc:205] Tried to move back new op lower bound from 7315361910893826048 to 7315361910700900352. Current Snapshot: MvccSnapshot[applied={T|T < 7315361910893826048}]
I20260806 00:43:49.037915 32469 tablet_replica.cc:439] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "175d3741a54b4588adc20c4f1fc915ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 } } peers { permanent_uuid: "175d3741a54b4588adc20c4f1fc915ce" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 41435 } } peers { permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 } } }
I20260806 00:43:49.037997 32469 tablet_replica.cc:442] T 06b1082b179b4d56a185fedf107a3d45 P 03b390384dec44d3b266c9842d093fc0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:49.047128 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:49.044085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:40251 (local address 127.29.84.130:41435)
0806 00:43:49.044202 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:49.044207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:49.044223 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:43:49.044369 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:49.044372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:49.044417 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:49.044496 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:43:49.044762 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.045220 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.046098 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.046317 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.046428 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:49.046669 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:43:49.046733 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:43:49.046808 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:49.046897 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 00:43:49.046966 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":52}
I20260806 00:43:49.047128 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:49.043723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:41435 (local address 127.29.84.130:40251)
0806 00:43:49.044163 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:49.044178 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:49.044268 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:49.044511 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:49.044514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:49.044520 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:43:49.044656 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:49.044662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.045343 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.045349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:49.045978 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:49.045987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.046420 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.046424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:49.046499 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:49.046547 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:43:49.046825 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:43:49.046831 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:43:49.046890 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 00:43:49.046979 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":27,"thread_start_us":151,"threads_started":1}
I20260806 00:43:49.053308 31945 raft_consensus.cc:1261] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Refusing update from remote peer 175d3741a54b4588adc20c4f1fc915ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:49.053320 32351 raft_consensus.cc:1261] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Refusing update from remote peer 175d3741a54b4588adc20c4f1fc915ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:49.053663 32525 consensus_queue.cc:1035] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "060a77fb83504edfbe7ddb7c56bebca9" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:49.053846 32472 consensus_queue.cc:1035] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "03b390384dec44d3b266c9842d093fc0" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 45625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:49.094497 31687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:49.094666 31687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:49.096725 31687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0755cfade724faca23a015c539cdf0e: Sending DeleteTablet for 3 replicas of tablet efdc876ef60c461c924a94e69a21e56a
I20260806 00:43:49.097404 31908 tablet_service.cc:1514] Processing DeleteTablet for tablet efdc876ef60c461c924a94e69a21e56a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:49 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:51442
I20260806 00:43:49.097498 32126 tablet_service.cc:1514] Processing DeleteTablet for tablet efdc876ef60c461c924a94e69a21e56a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:49 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:49390
I20260806 00:43:49.097591 32329 tablet_service.cc:1514] Processing DeleteTablet for tablet efdc876ef60c461c924a94e69a21e56a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:49 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:35414
I20260806 00:43:49.097693 32533 tablet_replica.cc:330] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: stopping tablet replica
I20260806 00:43:49.097761 32535 tablet_replica.cc:330] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: stopping tablet replica
I20260806 00:43:49.097805 32533 raft_consensus.cc:2229] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:49.097862 32535 raft_consensus.cc:2229] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:43:49.097923 32533 raft_consensus.cc:2258] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:49.098013 32535 raft_consensus.cc:2258] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:49.098239 32537 tablet_replica.cc:330] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: stopping tablet replica
I20260806 00:43:49.098336 32537 raft_consensus.cc:2229] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:49.098435 32537 raft_consensus.cc:2258] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:49.098903 32535 ts_tablet_manager.cc:1905] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:49.099568 32533 ts_tablet_manager.cc:1905] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:49.101889 32533 ts_tablet_manager.cc:1918] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:43:49.102002 32533 log.cc:1196] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/efdc876ef60c461c924a94e69a21e56a
I20260806 00:43:49.102029 32535 ts_tablet_manager.cc:1918] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:43:49.102113 32535 log.cc:1196] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/efdc876ef60c461c924a94e69a21e56a
I20260806 00:43:49.102279 32533 ts_tablet_manager.cc:1939] T efdc876ef60c461c924a94e69a21e56a P 03b390384dec44d3b266c9842d093fc0: Deleting consensus metadata
I20260806 00:43:49.102406 32535 ts_tablet_manager.cc:1939] T efdc876ef60c461c924a94e69a21e56a P 175d3741a54b4588adc20c4f1fc915ce: Deleting consensus metadata
I20260806 00:43:49.102957 31669 catalog_manager.cc:4841] TS 03b390384dec44d3b266c9842d093fc0 (127.29.84.129:45625): tablet efdc876ef60c461c924a94e69a21e56a (table test-table [id=36413f8de8324d2e94f4fed8802d0c5c]) successfully deleted
I20260806 00:43:49.103109 31668 catalog_manager.cc:4841] TS 175d3741a54b4588adc20c4f1fc915ce (127.29.84.130:41435): tablet efdc876ef60c461c924a94e69a21e56a (table test-table [id=36413f8de8324d2e94f4fed8802d0c5c]) successfully deleted
I20260806 00:43:49.103291 32537 ts_tablet_manager.cc:1905] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:49.105372 32537 ts_tablet_manager.cc:1918] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:43:49.105466 32537 log.cc:1196] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/efdc876ef60c461c924a94e69a21e56a
I20260806 00:43:49.105700 32537 ts_tablet_manager.cc:1939] T efdc876ef60c461c924a94e69a21e56a P 060a77fb83504edfbe7ddb7c56bebca9: Deleting consensus metadata
I20260806 00:43:49.106302 31669 catalog_manager.cc:4841] TS 060a77fb83504edfbe7ddb7c56bebca9 (127.29.84.131:39597): tablet efdc876ef60c461c924a94e69a21e56a (table test-table [id=36413f8de8324d2e94f4fed8802d0c5c]) successfully deleted
Aug 06 00:43:49 dist-test-slave-n326 krb5kdc[31623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977028, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:43:49.116016 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:49.099122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.29.84.129:45625)
0806 00:43:49.099659 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:49.099664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:49.106414 (+ 6750us) server_negotiation.cc:407] Connection header received
0806 00:43:49.106559 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:49.106562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:49.106608 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:49.106694 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:49.107440 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.108066 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.108882 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.109181 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.109279 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:49.112021 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:49.112030 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:49.112042 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:49.112063 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:49.113810 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:49.114300 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:49.114305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:49.114306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:49.114379 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:49.114685 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:49.114687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:49.114688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:49.114901 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:49.115049 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:49.115425 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 00:43:49.115566 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":115,"threads_started":1}
W20260806 00:43:49.116513 32004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51446
I20260806 00:43:49.125650 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:49.118209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44504 (local address 127.29.84.190:40969)
0806 00:43:49.118327 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:49.118331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:49.118390 (+ 59us) server_negotiation.cc:407] Connection header received
0806 00:43:49.118498 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:49.118501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:49.118548 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:49.118622 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:49.119186 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.119922 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.120629 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:49.120940 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:49.121030 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:49.122339 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:49.122343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:49.122345 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:49.122378 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:49.123955 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:49.124566 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:49.124569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:49.124571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:49.124657 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:49.124875 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:49.124878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:49.124879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:49.125070 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:49.125191 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:49.125399 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 00:43:49.125507 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 00:43:49.125965 31687 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44504
I20260806 00:43:49.131392 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 31802
I20260806 00:43:49.146464 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 32022
I20260806 00:43:49.156286 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 32239
I20260806 00:43:49.165712 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 31644
2026-08-06T00:43:49Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3022 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:43:49 dist-test-slave-n326 krb5kdc[32551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:43:49 dist-test-slave-n326 krb5kdc[32551](info): set up 2 sockets
Aug 06 00:43:49 dist-test-slave-n326 krb5kdc[32551](info): commencing operation
krb5kdc: starting...
W20260806 00:43:50.373610 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:43:50 dist-test-slave-n326 krb5kdc[32551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:43:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:43:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:43:50.494350 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:36329
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:50.599742 32575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:50.599984 32575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:50.600051 32575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:50.604058 32575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:43:50.604126 32575 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 00:43:50.604157 32575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:50.604212 32575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:50.604250 32575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:43:50.604285 32575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:43:50.609270 32575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:36329
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:50.610204 32575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:50.611202 32575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:50.618912 32584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:50.618981 32588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:50.619135 32575 server_base.cc:1034] running on GCE node
W20260806 00:43:50.618908 32591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:50.619690 32575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:50.620710 32575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:50.621872 32575 hybrid_clock.cc:648] HybridClock initialized: now 1785977030621853 us; error 29 us; skew 500 ppm
Aug 06 00:43:50 dist-test-slave-n326 krb5kdc[32551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977030, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:50.624812 32575 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:43:50.625674 32575 webserver.cc:460] Webserver started at http://127.29.84.190:35885/ using document root <none> and password file <none>
I20260806 00:43:50.626188 32575 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:50.626247 32575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:50.626471 32575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:50.628384 32575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "6cbb26b5d422470e8f192115a6fa1c56"
format_stamp: "Formatted at 2026-08-06 00:43:50 on dist-test-slave-n326"
server_key: "3193a4460fdcf3074bd03e19b11442a2"
server_key_iv: "d51aa56458995c5d36c94b1d710561a1"
server_key_version: "encryptionkey@0"
I20260806 00:43:50.628931 32575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "6cbb26b5d422470e8f192115a6fa1c56"
format_stamp: "Formatted at 2026-08-06 00:43:50 on dist-test-slave-n326"
server_key: "3193a4460fdcf3074bd03e19b11442a2"
server_key_iv: "d51aa56458995c5d36c94b1d710561a1"
server_key_version: "encryptionkey@0"
I20260806 00:43:50.632390 32575 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:43:50.634630 32600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:50.635779 32575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:50.635877 32575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "6cbb26b5d422470e8f192115a6fa1c56"
format_stamp: "Formatted at 2026-08-06 00:43:50 on dist-test-slave-n326"
server_key: "3193a4460fdcf3074bd03e19b11442a2"
server_key_iv: "d51aa56458995c5d36c94b1d710561a1"
server_key_version: "encryptionkey@0"
I20260806 00:43:50.636003 32575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:50.644696 32575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:50.647270 32575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:50.647451 32575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:50.654750 32575 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:36329
I20260806 00:43:50.654767 32686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:36329 every 8 connection(s)
I20260806 00:43:50.655628 32575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:43:50.658108 32687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:50.659392 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 32575
I20260806 00:43:50.659502 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:43:50.659745 30034 external_mini_cluster.cc:1428] Setting key 1bb98e6c25f6d92d61fa14339b3e6888
I20260806 00:43:50.663931 32687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56: Bootstrap starting.
I20260806 00:43:50.666321 32687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:50.667240 32687 log.cc:824] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:43:50 dist-test-slave-n326 krb5kdc[32551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:50.669067 32687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56: No bootstrap required, opened a new log
I20260806 00:43:50.671916 32687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb26b5d422470e8f192115a6fa1c56" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36329 } }
I20260806 00:43:50.672107 32687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:50.672230 32687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cbb26b5d422470e8f192115a6fa1c56, State: Initialized, Role: FOLLOWER
I20260806 00:43:50.672619 32687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb26b5d422470e8f192115a6fa1c56" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36329 } }
I20260806 00:43:50.672757 32687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:43:50.672821 32687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:43:50.672922 32687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:50.673789 32687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb26b5d422470e8f192115a6fa1c56" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36329 } }
I20260806 00:43:50.674110 32687 leader_election.cc:302] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cbb26b5d422470e8f192115a6fa1c56; no voters:
I20260806 00:43:50.674232 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:50.661087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.29.84.190:36329)
0806 00:43:50.661480 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:50.661496 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:50.661527 (+ 31us) server_negotiation.cc:407] Connection header received
0806 00:43:50.662114 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:50.662128 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:50.662412 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:50.662683 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:50.663600 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.664497 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.665663 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.666102 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.666221 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:50.668898 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:50.668903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:50.668914 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:50.668939 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:50.671287 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:50.671691 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:50.671696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:50.671700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:50.671796 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:50.672060 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:50.672063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:50.672065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:50.672539 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:50.672738 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:50.673315 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 00:43:50.673512 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":119,"threads_started":1}
I20260806 00:43:50.674432 32687 leader_election.cc:288] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:43:50.674587 32694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:50.674763 32694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [term 1 LEADER]: Becoming Leader. State: Replica: 6cbb26b5d422470e8f192115a6fa1c56, State: Running, Role: LEADER
I20260806 00:43:50.675016 32694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb26b5d422470e8f192115a6fa1c56" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36329 } }
I20260806 00:43:50.675629 32687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:43:50.676968 32695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cbb26b5d422470e8f192115a6fa1c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb26b5d422470e8f192115a6fa1c56" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36329 } } }
I20260806 00:43:50.677095 32695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [sys.catalog]: This master's current role is: LEADER
I20260806 00:43:50.677350 32696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cbb26b5d422470e8f192115a6fa1c56. Latest consensus state: current_term: 1 leader_uuid: "6cbb26b5d422470e8f192115a6fa1c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb26b5d422470e8f192115a6fa1c56" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36329 } } }
I20260806 00:43:50.677438 32705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:43:50.677466 32696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56 [sys.catalog]: This master's current role is: LEADER
I20260806 00:43:50.680480 32705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:43:50.685454 32705 catalog_manager.cc:1347] Generated new cluster ID: 7c7818c4bbbe44438fd3f16c5b85f26b
I20260806 00:43:50.685539 32705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:43:50.707093 32705 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:43:50.708128 32705 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:43:50.711939 32705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56: Generated new TSK 0
I20260806 00:43:50.712515 32705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:43:50.742769 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36329
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:50.850059 32724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:50.850312 32724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:50.850389 32724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:50.854257 32724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:50.854338 32724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:50.854408 32724 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 00:43:50.854451 32724 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:43:50.854511 32724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:43:50.859457 32724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.84.190:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:50.860350 32724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:50.861279 32724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:50.870402 32737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:50.870456 32724 server_base.cc:1034] running on GCE node
W20260806 00:43:50.870391 32734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:50.870393 32733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:50.871003 32724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:50.871623 32724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:50.872821 32724 hybrid_clock.cc:648] HybridClock initialized: now 1785977030872803 us; error 28 us; skew 500 ppm
Aug 06 00:43:50 dist-test-slave-n326 krb5kdc[32551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977030, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:50.875344 32724 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:43:50.876305 32724 webserver.cc:460] Webserver started at http://127.29.84.129:32847/ using document root <none> and password file <none>
I20260806 00:43:50.876911 32724 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:50.877034 32724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:50.877254 32724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:50.879194 32724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "65d31a838d804e838a454f579267c0ab"
format_stamp: "Formatted at 2026-08-06 00:43:50 on dist-test-slave-n326"
server_key: "cbeb3212d876697c91bdb6cd2c10b5f8"
server_key_iv: "3c0aa848795398bab8738cee419e2a67"
server_key_version: "encryptionkey@0"
I20260806 00:43:50.879716 32724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "65d31a838d804e838a454f579267c0ab"
format_stamp: "Formatted at 2026-08-06 00:43:50 on dist-test-slave-n326"
server_key: "cbeb3212d876697c91bdb6cd2c10b5f8"
server_key_iv: "3c0aa848795398bab8738cee419e2a67"
server_key_version: "encryptionkey@0"
I20260806 00:43:50.883152 32724 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:43:50.885424 32748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:50.886548 32724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:43:50.886703 32724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "65d31a838d804e838a454f579267c0ab"
format_stamp: "Formatted at 2026-08-06 00:43:50 on dist-test-slave-n326"
server_key: "cbeb3212d876697c91bdb6cd2c10b5f8"
server_key_iv: "3c0aa848795398bab8738cee419e2a67"
server_key_version: "encryptionkey@0"
I20260806 00:43:50.886811 32724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:50.908499 32724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:50.911162 32724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:50.911339 32724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:50.912616 32724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:50.912690 32724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:50.912751 32724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:50.912806 32724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:43:50 dist-test-slave-n326 krb5kdc[32551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977030, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:50.923812 32724 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:40489
I20260806 00:43:50.924463 446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:40489 every 8 connection(s)
I20260806 00:43:50.924680 32724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:43:50.926460 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:50.913422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:37343 (local address 127.29.84.190:36329)
0806 00:43:50.913545 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:50.913548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:50.914511 (+ 963us) server_negotiation.cc:407] Connection header received
0806 00:43:50.915484 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:50.915488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:50.915551 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:50.915653 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:50.917138 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.917841 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.919506 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.919740 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.919857 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:50.922653 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:50.922657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:50.922662 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:50.922707 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:50.924289 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:50.924863 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:50.924866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:50.924867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:50.924945 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:50.925348 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:50.925352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:50.925355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:50.925592 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:50.925709 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:50.926167 (+ 458us) server_negotiation.cc:299] Negotiation successful
0806 00:43:50.926291 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:43:50.927201 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:50.913896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36329 (local address 127.29.84.129:37343)
0806 00:43:50.914375 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:50.914414 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:50.915264 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:50.915856 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:50.915866 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:50.916342 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:50.916947 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:50.916966 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.917974 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.917978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:50.919364 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:50.919374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.919906 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.919911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:50.920071 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:50.920599 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:50.920614 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:50.922457 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:50.924405 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:50.924410 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:50.924422 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:50.924751 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:50.925042 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:50.925047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:50.925051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:50.925216 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:50.925708 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:50.925714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:50.925937 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 00:43:50.926120 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:43:50.926404 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":134,"threads_started":1}
I20260806 00:43:50.928570 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 32724
I20260806 00:43:50.928689 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:43:50.928943 30034 external_mini_cluster.cc:1428] Setting key e1c11838f25c4356bb979ce7063a9fd2
I20260806 00:43:50.930262 447 heartbeater.cc:347] Connected to a master server at 127.29.84.190:36329
I20260806 00:43:50.930389 447 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:50.930795 447 heartbeater.cc:511] Master 127.29.84.190:36329 requested a full tablet report, sending...
I20260806 00:43:50.932273 32620 ts_manager.cc:194] Registered new tserver with Master: 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489)
I20260806 00:43:50.933631 32620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:37343
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
I20260806 00:43:50.944172 456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:50.930511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36329 (local address 127.29.84.129:42753)
0806 00:43:50.930748 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:50.930762 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:50.930856 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:50.931159 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:50.931163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:50.931373 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:50.931526 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:50.931536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.932517 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.932520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:50.933326 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:50.933337 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.933955 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.933958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:50.934046 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:50.934652 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:50.934666 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:50.935046 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:50.942548 (+ 7502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:50.942555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:50.942559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:50.942825 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:50.943148 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:50.943150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:50.943152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:50.943234 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:50.943688 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:50.943690 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:50.943758 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 00:43:50.943831 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:43:50.943922 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260806 00:43:50.944252 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:50.930504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:42753 (local address 127.29.84.190:36329)
0806 00:43:50.930614 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:50.930618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:50.930847 (+ 229us) server_negotiation.cc:407] Connection header received
0806 00:43:50.930979 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:50.930984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:50.931026 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:50.931141 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:50.931693 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.932359 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.933458 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:50.933820 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:50.933951 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:50.935201 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:50.935207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:50.935212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:50.935243 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:50.942398 (+ 7155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:50.942934 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:50.942937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:50.942938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:50.943033 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:50.943329 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:50.943334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:50.943339 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:50.943582 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:50.943681 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:50.943928 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 00:43:50.944062 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:43:50.958431 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36329
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:51.067170 459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:51.067371 459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:51.067413 459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:51.070971 459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:51.071017 459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:51.071050 459 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 00:43:51.071113 459 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:43:51.071174 459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:43:51.076036 459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.84.190:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:51.076956 459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:51.077853 459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:51.086598 469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:51.086627 470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:51.086614 474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:51.086675 459 server_base.cc:1034] running on GCE node
I20260806 00:43:51.087343 459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:51.087890 459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:51.089107 459 hybrid_clock.cc:648] HybridClock initialized: now 1785977031089089 us; error 31 us; skew 500 ppm
Aug 06 00:43:51 dist-test-slave-n326 krb5kdc[32551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977031, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:51.095544 459 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:43:51.096331 459 webserver.cc:460] Webserver started at http://127.29.84.130:40333/ using document root <none> and password file <none>
I20260806 00:43:51.096839 459 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:51.096896 459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:51.097074 459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:51.098838 459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "f805208cd6004d468810764351bc299a"
format_stamp: "Formatted at 2026-08-06 00:43:51 on dist-test-slave-n326"
server_key: "c1c59e7b3ce62d5cb432f43acdf33600"
server_key_iv: "0b4aad78f5c34e010f824ba223bf770f"
server_key_version: "encryptionkey@0"
I20260806 00:43:51.099370 459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "f805208cd6004d468810764351bc299a"
format_stamp: "Formatted at 2026-08-06 00:43:51 on dist-test-slave-n326"
server_key: "c1c59e7b3ce62d5cb432f43acdf33600"
server_key_iv: "0b4aad78f5c34e010f824ba223bf770f"
server_key_version: "encryptionkey@0"
I20260806 00:43:51.102931 459 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260806 00:43:51.105052 486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:51.106237 459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:51.106365 459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "f805208cd6004d468810764351bc299a"
format_stamp: "Formatted at 2026-08-06 00:43:51 on dist-test-slave-n326"
server_key: "c1c59e7b3ce62d5cb432f43acdf33600"
server_key_iv: "0b4aad78f5c34e010f824ba223bf770f"
server_key_version: "encryptionkey@0"
I20260806 00:43:51.106485 459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:51.130630 459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:51.134364 459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:51.134562 459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:51.137887 459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:51.137945 459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:51.137998 459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:51.138031 459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:43:51 dist-test-slave-n326 krb5kdc[32551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977031, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:51.150144 459 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:38929
I20260806 00:43:51.150233 650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:38929 every 8 connection(s)
I20260806 00:43:51.151232 459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:43:51.153914 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 459
I20260806 00:43:51.154057 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:43:51.154268 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.139018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:53017 (local address 127.29.84.190:36329)
0806 00:43:51.139166 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.139170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.139186 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:43:51.139763 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.139766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.139850 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.139929 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:51.141292 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.142094 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.143933 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.144296 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.144426 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.148477 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:51.148482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:51.148485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:51.148517 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:51.150468 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.151279 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.151283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.151285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.151377 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.151824 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.151829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.151833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.152068 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:51.152150 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.153969 (+ 1819us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.154131 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 00:43:51.154449 30034 external_mini_cluster.cc:1428] Setting key ebefb45116cc07769e18de10e7d91c2a
I20260806 00:43:51.153833 498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.137343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36329 (local address 127.29.84.130:53017)
0806 00:43:51.137912 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.137964 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.139562 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.140125 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.140131 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.140537 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:51.141140 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.141164 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.142217 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.142221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.143796 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.143812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.144465 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.144471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.144701 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.145329 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:51.145354 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:51.148245 (+ 2891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:51.150598 (+ 2353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.150605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.150621 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.151159 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.151493 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.151496 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.151498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.151706 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.152201 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:51.152209 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:51.152475 (+ 266us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.152636 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.152934 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":131,"threads_started":1}
I20260806 00:43:51.156448 651 heartbeater.cc:347] Connected to a master server at 127.29.84.190:36329
I20260806 00:43:51.156596 651 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:51.157076 651 heartbeater.cc:511] Master 127.29.84.190:36329 requested a full tablet report, sending...
I20260806 00:43:51.158128 32621 ts_manager.cc:194] Registered new tserver with Master: f805208cd6004d468810764351bc299a (127.29.84.130:38929)
I20260806 00:43:51.158941 32621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:53017
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:43:51.167441 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.158831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:40693 (local address 127.29.84.190:36329)
0806 00:43:51.158948 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.158954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.159444 (+ 490us) server_negotiation.cc:407] Connection header received
0806 00:43:51.159590 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.159596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.159650 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.159701 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:51.160393 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.160862 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.161683 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.161917 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.162023 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.163446 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:51.163452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:51.163458 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:51.163490 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:51.165361 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.165949 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.165955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.165959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.166052 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.166420 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.166423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.166425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.166769 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:51.166883 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.167234 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.167325 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:43:51.167652 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.159060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36329 (local address 127.29.84.130:40693)
0806 00:43:51.159367 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.159381 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.159474 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.159842 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.159846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.160081 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:51.160266 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.160275 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.160981 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.160984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.161561 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.161569 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.162037 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.162040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.162136 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.162676 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:51.162696 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:51.163224 (+ 528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:51.165524 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.165528 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.165530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.165821 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.166155 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.166158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.166161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.166289 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.166891 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:51.166894 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:51.167011 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.167172 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.167349 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:43:51.186275 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36329
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:51.294721 663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:51.294934 663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:51.295002 663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:51.298799 663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:51.298846 663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:51.298920 663 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 00:43:51.298956 663 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:43:51.299045 663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:43:51.304091 663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.84.190:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:51.305023 663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:51.305935 663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:51.315109 668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:51.315142 669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:51.315189 663 server_base.cc:1034] running on GCE node
W20260806 00:43:51.315136 672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:51.315832 663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:51.316588 663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:51.317755 663 hybrid_clock.cc:648] HybridClock initialized: now 1785977031317723 us; error 54 us; skew 500 ppm
Aug 06 00:43:51 dist-test-slave-n326 krb5kdc[32551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977031, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:51.320441 663 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:43:51.321205 663 webserver.cc:460] Webserver started at http://127.29.84.131:46505/ using document root <none> and password file <none>
I20260806 00:43:51.321748 663 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:51.321841 663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:51.322073 663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:51.324005 663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "d0575f8ffb4f4e8283d09049cac1f85d"
format_stamp: "Formatted at 2026-08-06 00:43:51 on dist-test-slave-n326"
server_key: "d9bfa6c1b4b88d6cdb6cc996492baf7a"
server_key_iv: "c93af2555d50104c352916915f3b1bcb"
server_key_version: "encryptionkey@0"
I20260806 00:43:51.324525 663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "d0575f8ffb4f4e8283d09049cac1f85d"
format_stamp: "Formatted at 2026-08-06 00:43:51 on dist-test-slave-n326"
server_key: "d9bfa6c1b4b88d6cdb6cc996492baf7a"
server_key_iv: "c93af2555d50104c352916915f3b1bcb"
server_key_version: "encryptionkey@0"
I20260806 00:43:51.328161 663 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:43:51.330391 681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:51.331549 663 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:43:51.331678 663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "d0575f8ffb4f4e8283d09049cac1f85d"
format_stamp: "Formatted at 2026-08-06 00:43:51 on dist-test-slave-n326"
server_key: "d9bfa6c1b4b88d6cdb6cc996492baf7a"
server_key_iv: "c93af2555d50104c352916915f3b1bcb"
server_key_version: "encryptionkey@0"
I20260806 00:43:51.331784 663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:51.342161 663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:51.344745 663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:51.344950 663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:51.346210 663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:51.346284 663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:51.346352 663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:51.346407 663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:43:51 dist-test-slave-n326 krb5kdc[32551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977031, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:51.357710 663 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:45611
I20260806 00:43:51.357775 846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:45611 every 8 connection(s)
I20260806 00:43:51.358745 663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:43:51.360646 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.346983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:48041 (local address 127.29.84.190:36329)
0806 00:43:51.347141 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.347144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.348073 (+ 929us) server_negotiation.cc:407] Connection header received
0806 00:43:51.348929 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.348932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.348996 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.349101 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:51.350602 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.351461 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.353297 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.353682 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.353789 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.356815 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:51.356819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:51.356821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:51.356844 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:51.358632 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.359179 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.359184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.359188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.359283 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.359677 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.359681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.359683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.359929 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:51.360021 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.360438 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.360529 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:43:51.361301 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.347487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36329 (local address 127.29.84.131:48041)
0806 00:43:51.347933 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.347971 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.348730 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.349278 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.349287 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.349763 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:51.350445 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.350469 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.351614 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.351618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.353151 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.353168 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.353942 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.353945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.354070 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.354617 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:51.354641 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:51.356608 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:51.358747 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.358751 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.358761 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.358998 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.359392 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.359395 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.359397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.359557 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.360037 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:51.360041 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:51.360238 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.360372 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.360576 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":120,"threads_started":1}
I20260806 00:43:51.362294 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 663
I20260806 00:43:51.362421 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:43:51.362561 849 heartbeater.cc:347] Connected to a master server at 127.29.84.190:36329
I20260806 00:43:51.362722 849 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:51.362739 30034 external_mini_cluster.cc:1428] Setting key f3958ceb9e92a746f146e3bc63018550
I20260806 00:43:51.363255 849 heartbeater.cc:511] Master 127.29.84.190:36329 requested a full tablet report, sending...
I20260806 00:43:51.364272 32620 ts_manager.cc:194] Registered new tserver with Master: d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131:45611)
I20260806 00:43:51.364984 32620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:48041
I20260806 00:43:51.371398 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.364930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:39287 (local address 127.29.84.190:36329)
0806 00:43:51.365055 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.365060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.365341 (+ 281us) server_negotiation.cc:407] Connection header received
0806 00:43:51.365493 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.365495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.365554 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.365628 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:51.366027 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.366719 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.367577 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.367768 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.367845 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.368907 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:51.368910 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:51.368916 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:51.368933 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:51.370070 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.370424 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.370426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.370427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.370491 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.370750 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.370752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.370753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.370888 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:51.370965 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.371231 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.371308 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:43:51.371413 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.364963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36329 (local address 127.29.84.131:39287)
0806 00:43:51.365230 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.365246 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.365405 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.365656 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.365658 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.365811 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:51.365916 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.365921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.366840 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.366842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.367461 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.367470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.367862 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.367865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.367985 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.368502 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:51.368513 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:51.368799 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:51.370164 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.370166 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.370167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.370342 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.370605 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.370608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.370610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.370677 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.370971 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:51.370973 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:51.371038 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.371187 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.371284 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":111,"threads_started":1}
I20260806 00:43:51.376367 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:43:51 dist-test-slave-n326 krb5kdc[32551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:43:51 dist-test-slave-n326 krb5kdc[32551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:51.402017 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.394135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43982 (local address 127.29.84.190:36329)
0806 00:43:51.394264 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.394268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.394278 (+ 10us) server_negotiation.cc:407] Connection header received
0806 00:43:51.394307 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.394309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.394340 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.394424 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:51.395112 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.395617 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.396466 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.396683 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.396804 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.398992 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:51.398997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:51.399004 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:51.399026 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:51.400425 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.400835 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.400839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.400841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.400919 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.401195 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.401198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.401200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.401359 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:51.401524 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.401756 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.401861 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:43:51.404806 32620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:43:51.407735 32620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:43:51.420923 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.415678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:38929 (local address 127.0.0.1:43390)
0806 00:43:51.416153 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.416195 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.416339 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.417137 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.417142 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.417162 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:51.417326 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.417332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.418506 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.418510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.419526 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.419534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.420402 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.420406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.420515 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.420552 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.420657 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.420763 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":50,"thread_start_us":84,"threads_started":1}
I20260806 00:43:51.420938 881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.415800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43390 (local address 127.29.84.130:38929)
0806 00:43:51.416201 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.416213 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.416503 (+ 290us) server_negotiation.cc:407] Connection header received
0806 00:43:51.416588 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.416594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.416807 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.416970 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:51.417466 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.418405 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.419659 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.420282 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.420383 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.420540 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.420695 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.420804 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":86,"threads_started":1}
I20260806 00:43:51.421880 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.416371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45611 (local address 127.0.0.1:53486)
0806 00:43:51.417173 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.417185 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.417274 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.417931 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.417934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.417950 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:51.418025 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.418030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.419261 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.419263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.420407 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.420417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.421471 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.421475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.421561 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.421591 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.421658 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.421725 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":72,"thread_start_us":100,"threads_started":1}
I20260806 00:43:51.421911 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.416666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.29.84.131:45611)
0806 00:43:51.417526 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.417536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.417553 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:43:51.417607 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.417610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.417792 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.417949 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:51.418138 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.419153 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.420537 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.421314 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.421474 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.421648 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.421698 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.421778 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":121,"threads_started":1}
I20260806 00:43:51.422437 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.416888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55374 (local address 127.29.84.129:40489)
0806 00:43:51.417294 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.417307 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.417325 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:43:51.417381 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.417384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.417568 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.417723 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:51.417964 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.419032 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.420458 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.421250 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.421430 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.421597 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.422156 (+ 559us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.422257 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":96,"threads_started":1}
I20260806 00:43:51.422068 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.415786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:40489 (local address 127.0.0.1:55374)
0806 00:43:51.417030 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.417043 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.417132 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.417712 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.417714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.417734 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:51.417832 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.417842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.419188 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.419194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.420298 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.420308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.421661 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.421665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.421760 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.421780 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.421845 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.421910 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":37,"thread_start_us":112,"threads_started":1}
I20260806 00:43:51.424317 539 tablet_service.cc:1467] Processing CreateTablet for tablet 0d887a86788e45bf8218aa4673acdbf9 (DEFAULT_TABLE table=test-table [id=b18ba2bad58541b7af4eae138164feeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:51.424937 757 tablet_service.cc:1467] Processing CreateTablet for tablet 0d887a86788e45bf8218aa4673acdbf9 (DEFAULT_TABLE table=test-table [id=b18ba2bad58541b7af4eae138164feeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:51.425010 354 tablet_service.cc:1467] Processing CreateTablet for tablet 0d887a86788e45bf8218aa4673acdbf9 (DEFAULT_TABLE table=test-table [id=b18ba2bad58541b7af4eae138164feeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:51.425410 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d887a86788e45bf8218aa4673acdbf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:51.425897 757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d887a86788e45bf8218aa4673acdbf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:51.425917 354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d887a86788e45bf8218aa4673acdbf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:51.431761 886 tablet_bootstrap.cc:492] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: Bootstrap starting.
I20260806 00:43:51.433881 888 tablet_bootstrap.cc:492] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: Bootstrap starting.
I20260806 00:43:51.434402 886 tablet_bootstrap.cc:654] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:51.435222 886 log.cc:824] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:51.435384 889 tablet_bootstrap.cc:492] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: Bootstrap starting.
I20260806 00:43:51.435844 888 tablet_bootstrap.cc:654] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:51.436653 888 log.cc:824] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:51.437144 886 tablet_bootstrap.cc:492] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: No bootstrap required, opened a new log
I20260806 00:43:51.437393 886 ts_tablet_manager.cc:1397] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 00:43:51.437496 889 tablet_bootstrap.cc:654] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:51.438261 889 log.cc:824] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:51.438591 888 tablet_bootstrap.cc:492] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: No bootstrap required, opened a new log
I20260806 00:43:51.438818 888 ts_tablet_manager.cc:1397] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:43:51.440033 889 tablet_bootstrap.cc:492] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: No bootstrap required, opened a new log
I20260806 00:43:51.440229 889 ts_tablet_manager.cc:1397] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:43:51.441257 886 raft_consensus.cc:348] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.441541 886 raft_consensus.cc:374] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:51.441627 886 raft_consensus.cc:729] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d31a838d804e838a454f579267c0ab, State: Initialized, Role: FOLLOWER
I20260806 00:43:51.442082 886 consensus_queue.cc:260] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.442788 447 heartbeater.cc:503] Master 127.29.84.190:36329 was elected leader, sending a full tablet report...
I20260806 00:43:51.442689 888 raft_consensus.cc:348] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.442916 888 raft_consensus.cc:374] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:51.442972 888 raft_consensus.cc:729] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f805208cd6004d468810764351bc299a, State: Initialized, Role: FOLLOWER
I20260806 00:43:51.443397 888 consensus_queue.cc:260] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.443615 889 raft_consensus.cc:348] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.443907 889 raft_consensus.cc:374] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:51.443977 889 raft_consensus.cc:729] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0575f8ffb4f4e8283d09049cac1f85d, State: Initialized, Role: FOLLOWER
I20260806 00:43:51.444185 651 heartbeater.cc:503] Master 127.29.84.190:36329 was elected leader, sending a full tablet report...
I20260806 00:43:51.444358 888 ts_tablet_manager.cc:1428] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260806 00:43:51.444387 889 consensus_queue.cc:260] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.444967 849 heartbeater.cc:503] Master 127.29.84.190:36329 was elected leader, sending a full tablet report...
I20260806 00:43:51.445325 889 ts_tablet_manager.cc:1428] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:43:51.447329 886 ts_tablet_manager.cc:1428] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
W20260806 00:43:51.610038 850 tablet.cc:2369] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:43:51.654043 652 tablet.cc:2369] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:43:51.675987 448 tablet.cc:2369] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:43:51.679132 900 raft_consensus.cc:482] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:43:51.679287 900 raft_consensus.cc:504] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.680326 900 leader_election.cc:288] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f805208cd6004d468810764351bc299a (127.29.84.130:38929), 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489)
I20260806 00:43:51.683997 881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.680717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:37879 (local address 127.29.84.130:38929)
0806 00:43:51.680850 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.680854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.680869 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:43:51.680951 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.680953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.680992 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.681092 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:51.681416 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.681929 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.682976 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.683434 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.683562 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.683615 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.683815 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.683883 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:43:51.683997 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.680500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:38929 (local address 127.29.84.131:37879)
0806 00:43:51.680632 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.680647 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.680739 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.681106 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.681109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.681127 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:51.681253 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.681259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.682055 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.682060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.682834 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.682843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.683563 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.683566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.683680 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.683698 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.683784 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.683860 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 00:43:51.684621 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.680619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:40489 (local address 127.29.84.131:45133)
0806 00:43:51.680991 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.681001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.681113 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.681387 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.681389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.681405 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:51.681896 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.681903 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.682634 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.682638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.683416 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.683425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.684241 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.684246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.684338 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.684379 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.684450 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.684500 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":122,"threads_started":1}
I20260806 00:43:51.684702 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.680694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:45133 (local address 127.29.84.129:40489)
0806 00:43:51.680822 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.680827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.681087 (+ 260us) server_negotiation.cc:407] Connection header received
0806 00:43:51.681237 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.681240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.681280 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.681365 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:51.682032 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.682514 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.683532 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.684010 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.684168 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.684219 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.684494 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.684586 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:43:51.684674 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d887a86788e45bf8218aa4673acdbf9" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f805208cd6004d468810764351bc299a" is_pre_election: true
I20260806 00:43:51.684962 556 raft_consensus.cc:2454] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 0.
I20260806 00:43:51.685410 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d887a86788e45bf8218aa4673acdbf9" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab" is_pre_election: true
I20260806 00:43:51.685484 686 leader_election.cc:302] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0575f8ffb4f4e8283d09049cac1f85d, f805208cd6004d468810764351bc299a; no voters:
I20260806 00:43:51.685678 382 raft_consensus.cc:2454] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 0.
I20260806 00:43:51.685779 900 raft_consensus.cc:2781] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:43:51.685871 900 raft_consensus.cc:482] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:43:51.685926 900 raft_consensus.cc:3037] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:51.686964 900 raft_consensus.cc:504] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.687315 900 leader_election.cc:288] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 election: Requested vote from peers f805208cd6004d468810764351bc299a (127.29.84.130:38929), 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489)
I20260806 00:43:51.687664 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d887a86788e45bf8218aa4673acdbf9" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f805208cd6004d468810764351bc299a"
I20260806 00:43:51.687671 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d887a86788e45bf8218aa4673acdbf9" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab"
I20260806 00:43:51.687788 556 raft_consensus.cc:3037] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:51.687788 382 raft_consensus.cc:3037] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:51.689172 556 raft_consensus.cc:2454] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 1.
I20260806 00:43:51.689213 382 raft_consensus.cc:2454] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 1.
I20260806 00:43:51.689517 686 leader_election.cc:302] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0575f8ffb4f4e8283d09049cac1f85d, f805208cd6004d468810764351bc299a; no voters:
I20260806 00:43:51.689673 900 raft_consensus.cc:2781] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:51.689914 900 raft_consensus.cc:686] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 LEADER]: Becoming Leader. State: Replica: d0575f8ffb4f4e8283d09049cac1f85d, State: Running, Role: LEADER
I20260806 00:43:51.690237 900 consensus_queue.cc:237] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.693420 32620 catalog_manager.cc:5495] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d reported cstate change: term changed from 0 to 1, leader changed from <none> to d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131). New cstate: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } health_report { overall_health: HEALTHY } } }
Aug 06 00:43:51 dist-test-slave-n326 krb5kdc[32551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977030, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:51.732887 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.724624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36329 (local address 127.0.0.1:43988)
0806 00:43:51.724902 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.724914 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.725005 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.725205 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.725207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.725575 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:51.725665 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.725670 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.726407 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.726409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.726942 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.726948 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.727407 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.727409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.727476 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.728051 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:51.728063 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:51.729545 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:51.731325 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.731336 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.731341 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.731589 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.731873 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:51.731876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:51.731878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:51.731968 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:51.732333 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:51.732339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:51.732448 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.732562 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.732704 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":125,"threads_started":1}
I20260806 00:43:51.732887 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.724672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.29.84.190:36329)
0806 00:43:51.724804 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.724809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.724989 (+ 180us) server_negotiation.cc:407] Connection header received
0806 00:43:51.725064 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.725066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.725104 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.725167 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:51.725800 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.726311 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.727043 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.727309 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.727388 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.729685 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:51.729689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:51.729691 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:51.729717 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:51.731210 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.731689 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.731695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.731697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.731775 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:51.732067 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:51.732072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:51.732074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:51.732230 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:51.732339 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.732607 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.732754 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":52}
I20260806 00:43:51.736641 32620 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:43988:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 00:43:51.737267 32620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:43:51.741730 354 tablet_service.cc:1467] Processing CreateTablet for tablet 53d8bda942884c9e8ea840c871121274 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=59cf921a5f854ef4b27cc70c2477512a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:43:51.741730 539 tablet_service.cc:1467] Processing CreateTablet for tablet 53d8bda942884c9e8ea840c871121274 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=59cf921a5f854ef4b27cc70c2477512a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:43:51.741828 757 tablet_service.cc:1467] Processing CreateTablet for tablet 53d8bda942884c9e8ea840c871121274 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=59cf921a5f854ef4b27cc70c2477512a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:43:51.742453 354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d8bda942884c9e8ea840c871121274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:51.742453 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d8bda942884c9e8ea840c871121274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:51.742522 757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d8bda942884c9e8ea840c871121274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:51.745189 889 tablet_bootstrap.cc:492] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: Bootstrap starting.
I20260806 00:43:51.745340 886 tablet_bootstrap.cc:492] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: Bootstrap starting.
I20260806 00:43:51.745395 888 tablet_bootstrap.cc:492] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: Bootstrap starting.
I20260806 00:43:51.746166 886 tablet_bootstrap.cc:654] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:51.746254 889 tablet_bootstrap.cc:654] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:51.746289 888 tablet_bootstrap.cc:654] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:51.747371 886 tablet_bootstrap.cc:492] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: No bootstrap required, opened a new log
I20260806 00:43:51.747490 886 ts_tablet_manager.cc:1397] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:51.747792 888 tablet_bootstrap.cc:492] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: No bootstrap required, opened a new log
I20260806 00:43:51.747897 888 ts_tablet_manager.cc:1397] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 00:43:51.747893 886 raft_consensus.cc:348] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.748010 886 raft_consensus.cc:374] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:51.748060 886 raft_consensus.cc:729] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d31a838d804e838a454f579267c0ab, State: Initialized, Role: FOLLOWER
I20260806 00:43:51.748186 886 consensus_queue.cc:260] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.748400 888 raft_consensus.cc:348] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.748478 886 ts_tablet_manager.cc:1428] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:43:51.748435 898 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.748507 888 raft_consensus.cc:374] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:51.748541 888 raft_consensus.cc:729] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f805208cd6004d468810764351bc299a, State: Initialized, Role: FOLLOWER
I20260806 00:43:51.748562 898 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:51.748660 888 consensus_queue.cc:260] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.748917 888 ts_tablet_manager.cc:1428] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 00:43:51.749173 899 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.749320 899 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:51.750277 889 tablet_bootstrap.cc:492] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: No bootstrap required, opened a new log
I20260806 00:43:51.750365 889 ts_tablet_manager.cc:1397] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 00:43:51.750849 889 raft_consensus.cc:348] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.750960 889 raft_consensus.cc:374] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:51.751024 889 raft_consensus.cc:729] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0575f8ffb4f4e8283d09049cac1f85d, State: Initialized, Role: FOLLOWER
I20260806 00:43:51.751454 889 consensus_queue.cc:260] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.751736 900 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.751781 889 ts_tablet_manager.cc:1428] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:43:51.751878 900 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:51.816563 900 raft_consensus.cc:482] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:43:51.816700 900 raft_consensus.cc:504] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.817179 900 leader_election.cc:288] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489), f805208cd6004d468810764351bc299a (127.29.84.130:38929)
I20260806 00:43:51.817545 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d8bda942884c9e8ea840c871121274" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab" is_pre_election: true
I20260806 00:43:51.817600 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d8bda942884c9e8ea840c871121274" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f805208cd6004d468810764351bc299a" is_pre_election: true
I20260806 00:43:51.817708 382 raft_consensus.cc:2454] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 0.
I20260806 00:43:51.817765 556 raft_consensus.cc:2454] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 0.
I20260806 00:43:51.818008 688 leader_election.cc:302] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65d31a838d804e838a454f579267c0ab, d0575f8ffb4f4e8283d09049cac1f85d; no voters:
I20260806 00:43:51.818189 900 raft_consensus.cc:2781] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:43:51.818252 900 raft_consensus.cc:482] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:43:51.818295 900 raft_consensus.cc:3037] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:51.819281 900 raft_consensus.cc:504] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.819676 900 leader_election.cc:288] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 election: Requested vote from peers 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489), f805208cd6004d468810764351bc299a (127.29.84.130:38929)
I20260806 00:43:51.820015 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d8bda942884c9e8ea840c871121274" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab"
I20260806 00:43:51.820058 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d8bda942884c9e8ea840c871121274" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f805208cd6004d468810764351bc299a"
I20260806 00:43:51.820120 382 raft_consensus.cc:3037] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:51.820189 556 raft_consensus.cc:3037] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:51.821072 382 raft_consensus.cc:2454] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 1.
I20260806 00:43:51.821178 556 raft_consensus.cc:2454] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 1.
I20260806 00:43:51.821436 688 leader_election.cc:302] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65d31a838d804e838a454f579267c0ab, d0575f8ffb4f4e8283d09049cac1f85d; no voters:
I20260806 00:43:51.821686 900 raft_consensus.cc:2781] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:51.821782 900 raft_consensus.cc:686] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 LEADER]: Becoming Leader. State: Replica: d0575f8ffb4f4e8283d09049cac1f85d, State: Running, Role: LEADER
I20260806 00:43:51.821933 900 consensus_queue.cc:237] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.822619 900 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: TxnStatusTablet state changed. Reason: New leader d0575f8ffb4f4e8283d09049cac1f85d. Latest consensus state: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.822714 900 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:43:51.823032 916 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 00:43:51.823343 916 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 00:43:51.823712 32620 catalog_manager.cc:5495] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d reported cstate change: term changed from 0 to 1, leader changed from <none> to d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131). New cstate: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } health_report { overall_health: HEALTHY } } }
I20260806 00:43:51.850129 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.846464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45611 (local address 127.0.0.1:53500)
0806 00:43:51.846650 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.846666 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.846784 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.847177 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.847180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.847188 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:43:51.847289 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.847294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.848032 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.848037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.848635 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.848642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.849096 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.849099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.849189 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.849239 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:43:51.849843 (+ 604us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:43:51.849860 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.849925 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.850016 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 00:43:51.850129 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.846577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.29.84.131:45611)
0806 00:43:51.846687 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.846694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.846720 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:43:51.846968 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.846970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.847020 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.847160 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:43:51.847407 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.847917 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.848755 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.848986 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.849094 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.849403 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:43:51.849742 (+ 339us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:43:51.849863 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.849952 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.850004 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:43:51.855295 556 raft_consensus.cc:1261] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Refusing update from remote peer d0575f8ffb4f4e8283d09049cac1f85d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:51.855325 382 raft_consensus.cc:1261] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Refusing update from remote peer d0575f8ffb4f4e8283d09049cac1f85d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:51.855947 900 consensus_queue.cc:1035] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:51.856137 902 consensus_queue.cc:1035] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d [LEADER]: Connected to new peer: Peer: permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:51.860071 899 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: TxnStatusTablet state changed. Reason: New leader d0575f8ffb4f4e8283d09049cac1f85d. Latest consensus state: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.860193 899 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:51.859745 898 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: TxnStatusTablet state changed. Reason: New leader d0575f8ffb4f4e8283d09049cac1f85d. Latest consensus state: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.860402 898 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:51.863862 900 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.863996 900 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:43:51.864090 923 mvcc.cc:205] Tried to move back new op lower bound from 7315361922472706048 to 7315361922344034304. Current Snapshot: MvccSnapshot[applied={T|T < 7315361922472706048}]
I20260806 00:43:51.864765 899 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.864868 899 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P f805208cd6004d468810764351bc299a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:51.865463 902 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.865564 902 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P d0575f8ffb4f4e8283d09049cac1f85d: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:43:51.865840 924 mvcc.cc:205] Tried to move back new op lower bound from 7315361922472706048 to 7315361922344034304. Current Snapshot: MvccSnapshot[applied={T|T < 7315361922472706048}]
I20260806 00:43:51.867056 898 tablet_replica.cc:439] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } } }
I20260806 00:43:51.867184 898 tablet_replica.cc:442] T 53d8bda942884c9e8ea840c871121274 P 65d31a838d804e838a454f579267c0ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:43:51.869273 925 mvcc.cc:205] Tried to move back new op lower bound from 7315361922472706048 to 7315361922344034304. Current Snapshot: MvccSnapshot[applied={T|T < 7315361922472706048}]
I20260806 00:43:51.873296 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.869705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.29.84.131:45611)
0806 00:43:51.869809 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.869813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.869824 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:43:51.869860 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.869863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.869899 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.869990 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:43:51.870265 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.870981 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.872196 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.872463 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.872563 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.872887 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:43:51.872961 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:43:51.873039 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.873125 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.873170 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 00:43:51.883025 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.879751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:51549 (local address 127.29.84.131:45611)
0806 00:43:51.879860 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:51.879864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:51.880059 (+ 195us) server_negotiation.cc:407] Connection header received
0806 00:43:51.880159 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:51.880161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:51.880194 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:51.880248 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:43:51.880480 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.880871 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.882067 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.882260 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.882335 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.882679 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:43:51.882730 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:43:51.882778 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:51.882888 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 00:43:51.882923 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:43:51.883163 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:51.879710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45611 (local address 127.29.84.131:51549)
0806 00:43:51.879974 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:51.879988 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:51.880081 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:51.880278 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:51.880281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:51.880287 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:43:51.880384 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.880389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.880960 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.880963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.881948 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:51.881960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:51.882363 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:51.882367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:51.882468 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:51.882516 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:43:51.882805 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:43:51.882811 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:43:51.882871 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 00:43:51.882946 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260806 00:43:51.889310 382 raft_consensus.cc:1261] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Refusing update from remote peer d0575f8ffb4f4e8283d09049cac1f85d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:51.889333 556 raft_consensus.cc:1261] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Refusing update from remote peer d0575f8ffb4f4e8283d09049cac1f85d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:51.889686 937 consensus_queue.cc:1035] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:51.889966 900 consensus_queue.cc:1035] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [LEADER]: Connected to new peer: Peer: permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:51.930186 32620 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:51.930347 32620 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:51.932689 32620 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56: Sending DeleteTablet for 3 replicas of tablet 0d887a86788e45bf8218aa4673acdbf9
I20260806 00:43:51.933266 539 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d887a86788e45bf8218aa4673acdbf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:51 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:43390
I20260806 00:43:51.933372 354 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d887a86788e45bf8218aa4673acdbf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:51 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:55374
I20260806 00:43:51.933660 756 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d887a86788e45bf8218aa4673acdbf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:51 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53486
I20260806 00:43:51.933810 950 tablet_replica.cc:330] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: stopping tablet replica
I20260806 00:43:51.934059 950 raft_consensus.cc:2229] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:51.934226 950 raft_consensus.cc:2258] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:51.934326 32620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:43:51.934712 32620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:43:51.935482 950 ts_tablet_manager.cc:1905] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:51.938298 950 ts_tablet_manager.cc:1918] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:43:51.938401 950 log.cc:1196] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/0d887a86788e45bf8218aa4673acdbf9
I20260806 00:43:51.938766 950 ts_tablet_manager.cc:1939] T 0d887a86788e45bf8218aa4673acdbf9 P 65d31a838d804e838a454f579267c0ab: Deleting consensus metadata
I20260806 00:43:51.940280 539 tablet_service.cc:1467] Processing CreateTablet for tablet f298f37a05b34e43a46d0dee0f6c578c (DEFAULT_TABLE table=test-table [id=c2386bafcbf944f69969d4716c700b47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:51.940536 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f298f37a05b34e43a46d0dee0f6c578c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:51.940544 32605 catalog_manager.cc:4841] TS 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489): tablet 0d887a86788e45bf8218aa4673acdbf9 (table test-table [id=b18ba2bad58541b7af4eae138164feeb]) successfully deleted
I20260806 00:43:51.940964 354 tablet_service.cc:1467] Processing CreateTablet for tablet f298f37a05b34e43a46d0dee0f6c578c (DEFAULT_TABLE table=test-table [id=c2386bafcbf944f69969d4716c700b47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:51.941183 354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f298f37a05b34e43a46d0dee0f6c578c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:51.941428 949 tablet_replica.cc:330] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: stopping tablet replica
I20260806 00:43:51.941706 949 raft_consensus.cc:2229] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:51.941905 949 raft_consensus.cc:2258] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:51.942771 888 tablet_bootstrap.cc:492] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: Bootstrap starting.
I20260806 00:43:51.943733 886 tablet_bootstrap.cc:492] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: Bootstrap starting.
I20260806 00:43:51.943800 888 tablet_bootstrap.cc:654] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:51.943979 951 tablet_replica.cc:330] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: stopping tablet replica
I20260806 00:43:51.944258 951 raft_consensus.cc:2229] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:43:51.944617 951 raft_consensus.cc:2258] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:51.944648 886 tablet_bootstrap.cc:654] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:51.945366 949 ts_tablet_manager.cc:1905] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:51.945513 756 tablet_service.cc:1467] Processing CreateTablet for tablet f298f37a05b34e43a46d0dee0f6c578c (DEFAULT_TABLE table=test-table [id=c2386bafcbf944f69969d4716c700b47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:51.945806 756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f298f37a05b34e43a46d0dee0f6c578c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:51.945827 951 ts_tablet_manager.cc:1905] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:51.948112 949 ts_tablet_manager.cc:1918] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:43:51.948210 949 log.cc:1196] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/0d887a86788e45bf8218aa4673acdbf9
I20260806 00:43:51.948553 949 ts_tablet_manager.cc:1939] T 0d887a86788e45bf8218aa4673acdbf9 P f805208cd6004d468810764351bc299a: Deleting consensus metadata
I20260806 00:43:51.948966 889 tablet_bootstrap.cc:492] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: Bootstrap starting.
I20260806 00:43:51.949386 951 ts_tablet_manager.cc:1918] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:43:51.949550 951 log.cc:1196] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/0d887a86788e45bf8218aa4673acdbf9
I20260806 00:43:51.949864 889 tablet_bootstrap.cc:654] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:51.949829 32602 catalog_manager.cc:4841] TS f805208cd6004d468810764351bc299a (127.29.84.130:38929): tablet 0d887a86788e45bf8218aa4673acdbf9 (table test-table [id=b18ba2bad58541b7af4eae138164feeb]) successfully deleted
I20260806 00:43:51.950084 951 ts_tablet_manager.cc:1939] T 0d887a86788e45bf8218aa4673acdbf9 P d0575f8ffb4f4e8283d09049cac1f85d: Deleting consensus metadata
I20260806 00:43:51.950695 888 tablet_bootstrap.cc:492] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: No bootstrap required, opened a new log
I20260806 00:43:51.950793 888 ts_tablet_manager.cc:1397] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 00:43:51.951284 32605 catalog_manager.cc:4841] TS d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131:45611): tablet 0d887a86788e45bf8218aa4673acdbf9 (table test-table [id=b18ba2bad58541b7af4eae138164feeb]) successfully deleted
I20260806 00:43:51.951248 888 raft_consensus.cc:348] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.951366 888 raft_consensus.cc:374] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:51.951411 888 raft_consensus.cc:729] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f805208cd6004d468810764351bc299a, State: Initialized, Role: FOLLOWER
I20260806 00:43:51.951542 888 consensus_queue.cc:260] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.951848 888 ts_tablet_manager.cc:1428] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:43:51.952061 889 tablet_bootstrap.cc:492] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: No bootstrap required, opened a new log
I20260806 00:43:51.952150 889 ts_tablet_manager.cc:1397] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:43:51.952570 889 raft_consensus.cc:348] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.952677 889 raft_consensus.cc:374] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:51.952726 889 raft_consensus.cc:729] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0575f8ffb4f4e8283d09049cac1f85d, State: Initialized, Role: FOLLOWER
I20260806 00:43:51.952857 889 consensus_queue.cc:260] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.953112 889 ts_tablet_manager.cc:1428] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:43:51.955812 886 tablet_bootstrap.cc:492] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: No bootstrap required, opened a new log
I20260806 00:43:51.955922 886 ts_tablet_manager.cc:1397] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260806 00:43:51.956354 886 raft_consensus.cc:348] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.956517 886 raft_consensus.cc:374] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:51.956557 886 raft_consensus.cc:729] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d31a838d804e838a454f579267c0ab, State: Initialized, Role: FOLLOWER
I20260806 00:43:51.956677 886 consensus_queue.cc:260] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:51.956940 886 ts_tablet_manager.cc:1428] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:43:52.098443 947 raft_consensus.cc:482] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:43:52.098594 947 raft_consensus.cc:504] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:52.099138 947 leader_election.cc:288] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489), f805208cd6004d468810764351bc299a (127.29.84.130:38929)
I20260806 00:43:52.099560 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f298f37a05b34e43a46d0dee0f6c578c" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab" is_pre_election: true
I20260806 00:43:52.099581 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f298f37a05b34e43a46d0dee0f6c578c" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f805208cd6004d468810764351bc299a" is_pre_election: true
I20260806 00:43:52.099728 382 raft_consensus.cc:2454] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 0.
I20260806 00:43:52.099772 556 raft_consensus.cc:2454] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 0.
I20260806 00:43:52.100090 686 leader_election.cc:302] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0575f8ffb4f4e8283d09049cac1f85d, f805208cd6004d468810764351bc299a; no voters:
I20260806 00:43:52.100283 947 raft_consensus.cc:2781] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:43:52.100342 947 raft_consensus.cc:482] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:43:52.100381 947 raft_consensus.cc:3037] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:52.101415 947 raft_consensus.cc:504] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:52.101722 947 leader_election.cc:288] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 election: Requested vote from peers 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489), f805208cd6004d468810764351bc299a (127.29.84.130:38929)
I20260806 00:43:52.102005 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f298f37a05b34e43a46d0dee0f6c578c" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab"
I20260806 00:43:52.102159 382 raft_consensus.cc:3037] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:52.102180 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f298f37a05b34e43a46d0dee0f6c578c" candidate_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f805208cd6004d468810764351bc299a"
I20260806 00:43:52.102290 556 raft_consensus.cc:3037] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:52.103338 382 raft_consensus.cc:2454] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 1.
I20260806 00:43:52.103451 556 raft_consensus.cc:2454] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0575f8ffb4f4e8283d09049cac1f85d in term 1.
I20260806 00:43:52.103798 688 leader_election.cc:302] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65d31a838d804e838a454f579267c0ab, d0575f8ffb4f4e8283d09049cac1f85d; no voters:
I20260806 00:43:52.104000 947 raft_consensus.cc:2781] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:52.104112 947 raft_consensus.cc:686] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 1 LEADER]: Becoming Leader. State: Replica: d0575f8ffb4f4e8283d09049cac1f85d, State: Running, Role: LEADER
I20260806 00:43:52.104271 947 consensus_queue.cc:237] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:52.106047 32621 catalog_manager.cc:5495] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d reported cstate change: term changed from 0 to 1, leader changed from <none> to d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131). New cstate: current_term: 1 leader_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } health_report { overall_health: HEALTHY } } }
I20260806 00:43:52.149997 382 raft_consensus.cc:1261] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Refusing update from remote peer d0575f8ffb4f4e8283d09049cac1f85d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:52.149883 556 raft_consensus.cc:1261] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Refusing update from remote peer d0575f8ffb4f4e8283d09049cac1f85d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:52.151413 900 consensus_queue.cc:1035] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:52.151600 947 consensus_queue.cc:1035] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [LEADER]: Connected to new peer: Peer: permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:43:52.165555 923 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 00:43:53.714402 32621 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:53.714543 32621 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:53.716748 32621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56: Sending DeleteTablet for 3 replicas of tablet f298f37a05b34e43a46d0dee0f6c578c
I20260806 00:43:53.717296 354 tablet_service.cc:1514] Processing DeleteTablet for tablet f298f37a05b34e43a46d0dee0f6c578c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:53 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:55374
I20260806 00:43:53.717310 539 tablet_service.cc:1514] Processing DeleteTablet for tablet f298f37a05b34e43a46d0dee0f6c578c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:53 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:43390
I20260806 00:43:53.717628 757 tablet_service.cc:1514] Processing DeleteTablet for tablet f298f37a05b34e43a46d0dee0f6c578c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:53 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53486
I20260806 00:43:53.717689 989 tablet_replica.cc:330] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: stopping tablet replica
I20260806 00:43:53.717809 989 raft_consensus.cc:2229] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:53.717917 989 raft_consensus.cc:2258] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:53.718261 987 tablet_replica.cc:330] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: stopping tablet replica
I20260806 00:43:53.718382 987 raft_consensus.cc:2229] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:53.718474 987 raft_consensus.cc:2258] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:53.718547 989 ts_tablet_manager.cc:1905] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:53.718997 987 ts_tablet_manager.cc:1905] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:53.721586 989 ts_tablet_manager.cc:1918] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:43:53.721706 987 ts_tablet_manager.cc:1918] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:43:53.721786 987 log.cc:1196] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/f298f37a05b34e43a46d0dee0f6c578c
I20260806 00:43:53.721736 989 log.cc:1196] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/f298f37a05b34e43a46d0dee0f6c578c
I20260806 00:43:53.722077 987 ts_tablet_manager.cc:1939] T f298f37a05b34e43a46d0dee0f6c578c P 65d31a838d804e838a454f579267c0ab: Deleting consensus metadata
I20260806 00:43:53.722266 989 ts_tablet_manager.cc:1939] T f298f37a05b34e43a46d0dee0f6c578c P f805208cd6004d468810764351bc299a: Deleting consensus metadata
I20260806 00:43:53.722774 32605 catalog_manager.cc:4841] TS 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489): tablet f298f37a05b34e43a46d0dee0f6c578c (table test-table [id=c2386bafcbf944f69969d4716c700b47]) successfully deleted
I20260806 00:43:53.722899 32602 catalog_manager.cc:4841] TS f805208cd6004d468810764351bc299a (127.29.84.130:38929): tablet f298f37a05b34e43a46d0dee0f6c578c (table test-table [id=c2386bafcbf944f69969d4716c700b47]) successfully deleted
I20260806 00:43:53.723117 991 tablet_replica.cc:330] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: stopping tablet replica
I20260806 00:43:53.723258 991 raft_consensus.cc:2229] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:43:53.723448 991 raft_consensus.cc:2258] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:53.723920 991 ts_tablet_manager.cc:1905] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:53.726130 991 ts_tablet_manager.cc:1918] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:43:53.726197 991 log.cc:1196] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/f298f37a05b34e43a46d0dee0f6c578c
I20260806 00:43:53.726500 991 ts_tablet_manager.cc:1939] T f298f37a05b34e43a46d0dee0f6c578c P d0575f8ffb4f4e8283d09049cac1f85d: Deleting consensus metadata
I20260806 00:43:53.727169 32605 catalog_manager.cc:4841] TS d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131:45611): tablet f298f37a05b34e43a46d0dee0f6c578c (table test-table [id=c2386bafcbf944f69969d4716c700b47]) successfully deleted
Aug 06 00:43:53 dist-test-slave-n326 krb5kdc[32551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:43:53 dist-test-slave-n326 krb5kdc[32551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:53.746364 1001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.737872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43994 (local address 127.29.84.190:36329)
0806 00:43:53.738104 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:53.738111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:53.738215 (+ 104us) server_negotiation.cc:407] Connection header received
0806 00:43:53.738314 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:53.738316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:53.738357 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:53.738432 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:53.739261 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.739883 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.740675 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.740906 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.740982 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.743440 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:53.743443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:53.743445 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:53.743483 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:53.745026 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:53.745379 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:53.745382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:53.745383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:53.745472 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:53.745709 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:53.745711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:53.745712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:53.745868 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:53.745966 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:53.746157 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 00:43:53.746248 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
I20260806 00:43:53.748904 32621 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:43:53.749264 32621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:43:53.753504 354 tablet_service.cc:1467] Processing CreateTablet for tablet e146ec596b5841959ffabc6bc2f38e4a (DEFAULT_TABLE table=test-table [id=b7652f7c534346a5b8dc769d896557b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:53.753593 539 tablet_service.cc:1467] Processing CreateTablet for tablet e146ec596b5841959ffabc6bc2f38e4a (DEFAULT_TABLE table=test-table [id=b7652f7c534346a5b8dc769d896557b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:53.753674 757 tablet_service.cc:1467] Processing CreateTablet for tablet e146ec596b5841959ffabc6bc2f38e4a (DEFAULT_TABLE table=test-table [id=b7652f7c534346a5b8dc769d896557b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:53.753785 354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e146ec596b5841959ffabc6bc2f38e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:53.753898 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e146ec596b5841959ffabc6bc2f38e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:53.753929 757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e146ec596b5841959ffabc6bc2f38e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:53.755889 1002 tablet_bootstrap.cc:492] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: Bootstrap starting.
I20260806 00:43:53.756471 1003 tablet_bootstrap.cc:492] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: Bootstrap starting.
I20260806 00:43:53.757249 1003 tablet_bootstrap.cc:654] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:53.757280 1005 tablet_bootstrap.cc:492] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: Bootstrap starting.
I20260806 00:43:53.757900 1002 tablet_bootstrap.cc:654] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:53.758571 1005 tablet_bootstrap.cc:654] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:53.758925 1003 tablet_bootstrap.cc:492] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: No bootstrap required, opened a new log
I20260806 00:43:53.759048 1003 ts_tablet_manager.cc:1397] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:43:53.759532 1003 raft_consensus.cc:348] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.759691 1003 raft_consensus.cc:374] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:53.759784 1003 raft_consensus.cc:729] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d31a838d804e838a454f579267c0ab, State: Initialized, Role: FOLLOWER
I20260806 00:43:53.759876 1002 tablet_bootstrap.cc:492] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: No bootstrap required, opened a new log
I20260806 00:43:53.759979 1002 ts_tablet_manager.cc:1397] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 00:43:53.759955 1003 consensus_queue.cc:260] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.760303 1005 tablet_bootstrap.cc:492] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: No bootstrap required, opened a new log
I20260806 00:43:53.760401 1003 ts_tablet_manager.cc:1428] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:43:53.760435 1005 ts_tablet_manager.cc:1397] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260806 00:43:53.760433 1002 raft_consensus.cc:348] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.760598 1002 raft_consensus.cc:374] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:53.760679 1002 raft_consensus.cc:729] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f805208cd6004d468810764351bc299a, State: Initialized, Role: FOLLOWER
I20260806 00:43:53.760864 1002 consensus_queue.cc:260] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.760959 1005 raft_consensus.cc:348] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.761096 1005 raft_consensus.cc:374] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:53.761154 1005 raft_consensus.cc:729] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0575f8ffb4f4e8283d09049cac1f85d, State: Initialized, Role: FOLLOWER
I20260806 00:43:53.761293 1002 ts_tablet_manager.cc:1428] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:43:53.761304 1005 consensus_queue.cc:260] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.762375 1005 ts_tablet_manager.cc:1428] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 00:43:53.868263 1010 raft_consensus.cc:482] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:43:53.868395 1010 raft_consensus.cc:504] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.869329 1010 leader_election.cc:288] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489), d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131:45611)
I20260806 00:43:53.874136 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.869558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:40489 (local address 127.29.84.130:40611)
0806 00:43:53.869887 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:53.869899 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:53.870009 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:53.870311 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:53.870314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:53.870335 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:53.870524 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:53.870530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.871675 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.871681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:53.872632 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:53.872642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.873453 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.873456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:53.873701 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.873732 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 00:43:53.873821 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 00:43:53.873897 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":148,"threads_started":1}
I20260806 00:43:53.874157 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.869697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:40611 (local address 127.29.84.129:40489)
0806 00:43:53.869988 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:53.869996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:53.870013 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:43:53.870166 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:53.870168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:53.870208 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:53.870295 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:53.870649 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.871557 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.872881 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.873327 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.873608 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.873689 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:53.873851 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 00:43:53.873922 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
I20260806 00:43:53.874167 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.870137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:33397 (local address 127.29.84.131:45611)
0806 00:43:53.870429 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:53.870440 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:53.870561 (+ 121us) server_negotiation.cc:407] Connection header received
0806 00:43:53.870727 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:53.870729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:53.870767 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:53.870853 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:43:53.871345 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.871871 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.873143 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.873543 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.873666 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.873715 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:53.873964 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 00:43:53.874048 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":87,"threads_started":1}
I20260806 00:43:53.874236 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.870021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45611 (local address 127.29.84.130:33397)
0806 00:43:53.870429 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:53.870442 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:53.870600 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:53.870865 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:53.870868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:53.870891 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:43:53.871213 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:53.871220 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.872096 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.872099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:53.873013 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:53.873023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.873666 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.873669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:53.873817 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.873833 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 00:43:53.873946 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 00:43:53.874013 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":100,"threads_started":1}
I20260806 00:43:53.874686 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e146ec596b5841959ffabc6bc2f38e4a" candidate_uuid: "f805208cd6004d468810764351bc299a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab" is_pre_election: true
I20260806 00:43:53.874830 382 raft_consensus.cc:2454] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f805208cd6004d468810764351bc299a in term 0.
I20260806 00:43:53.875057 783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e146ec596b5841959ffabc6bc2f38e4a" candidate_uuid: "f805208cd6004d468810764351bc299a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" is_pre_election: true
I20260806 00:43:53.875334 783 raft_consensus.cc:2454] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f805208cd6004d468810764351bc299a in term 0.
I20260806 00:43:53.875370 491 leader_election.cc:302] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65d31a838d804e838a454f579267c0ab, f805208cd6004d468810764351bc299a; no voters:
I20260806 00:43:53.875648 1010 raft_consensus.cc:2781] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:43:53.875701 1010 raft_consensus.cc:482] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:43:53.875792 1010 raft_consensus.cc:3037] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:53.876925 1010 raft_consensus.cc:504] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.877257 1010 leader_election.cc:288] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [CANDIDATE]: Term 1 election: Requested vote from peers 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489), d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131:45611)
I20260806 00:43:53.877640 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e146ec596b5841959ffabc6bc2f38e4a" candidate_uuid: "f805208cd6004d468810764351bc299a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab"
I20260806 00:43:53.877650 783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e146ec596b5841959ffabc6bc2f38e4a" candidate_uuid: "f805208cd6004d468810764351bc299a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0575f8ffb4f4e8283d09049cac1f85d"
I20260806 00:43:53.877774 382 raft_consensus.cc:3037] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:53.877764 783 raft_consensus.cc:3037] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:53.878736 382 raft_consensus.cc:2454] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f805208cd6004d468810764351bc299a in term 1.
I20260806 00:43:53.878897 783 raft_consensus.cc:2454] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f805208cd6004d468810764351bc299a in term 1.
I20260806 00:43:53.879016 491 leader_election.cc:302] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65d31a838d804e838a454f579267c0ab, f805208cd6004d468810764351bc299a; no voters:
I20260806 00:43:53.879215 1010 raft_consensus.cc:2781] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:53.879424 1010 raft_consensus.cc:686] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 1 LEADER]: Becoming Leader. State: Replica: f805208cd6004d468810764351bc299a, State: Running, Role: LEADER
I20260806 00:43:53.879784 1010 consensus_queue.cc:237] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.881922 32619 catalog_manager.cc:5495] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a reported cstate change: term changed from 0 to 1, leader changed from <none> to f805208cd6004d468810764351bc299a (127.29.84.130). New cstate: current_term: 1 leader_uuid: "f805208cd6004d468810764351bc299a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } health_report { overall_health: UNKNOWN } } }
I20260806 00:43:53.917852 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.913567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43396 (local address 127.29.84.130:38929)
0806 00:43:53.913867 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:53.913913 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:53.913926 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:43:53.913992 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:53.913994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:53.914042 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:53.914169 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:43:53.914449 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.915249 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.916109 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.916429 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.916548 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.916828 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:43:53.917268 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:43:53.917401 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:53.917473 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 00:43:53.917522 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":116,"threads_started":1}
I20260806 00:43:53.928181 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.924501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:51283 (local address 127.29.84.131:45611)
0806 00:43:53.924664 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:53.924670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:53.925014 (+ 344us) server_negotiation.cc:407] Connection header received
0806 00:43:53.925129 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:53.925131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:53.925169 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:53.925229 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:43:53.925478 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.925960 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.927152 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.927355 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.927447 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.927796 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:43:53.927864 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:43:53.927919 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:53.928041 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 00:43:53.928083 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:43:53.928238 1033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.924501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45611 (local address 127.29.84.130:51283)
0806 00:43:53.924924 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:53.924939 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:53.925038 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:53.925260 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:53.925263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:53.925270 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:43:53.925384 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:53.925389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.926055 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.926058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:53.927016 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:53.927026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.927490 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.927494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:53.927590 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.927636 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:43:53.927947 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:43:53.927953 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:43:53.928020 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 00:43:53.928092 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":150,"threads_started":1}
I20260806 00:43:53.933547 382 raft_consensus.cc:1261] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Refusing update from remote peer f805208cd6004d468810764351bc299a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:53.933626 783 raft_consensus.cc:1261] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Refusing update from remote peer f805208cd6004d468810764351bc299a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:53.934056 1010 consensus_queue.cc:1035] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [LEADER]: Connected to new peer: Peer: permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:53.934337 1021 consensus_queue.cc:1035] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:53.951570 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.948198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:48025 (local address 127.29.84.130:38929)
0806 00:43:53.948332 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:53.948336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:53.948608 (+ 272us) server_negotiation.cc:407] Connection header received
0806 00:43:53.948665 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:53.948668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:53.948722 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:53.948813 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:43:53.948972 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.949429 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.950379 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.950658 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.950789 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.951085 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:43:53.951160 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:43:53.951270 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:53.951413 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 00:43:53.951459 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:43:53.951611 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:53.948059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:38929 (local address 127.29.84.131:48025)
0806 00:43:53.948425 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:53.948439 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:53.948579 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:53.948823 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:53.948825 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:53.948832 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:43:53.948907 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:53.948911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.949515 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.949517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:53.950218 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:53.950231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:53.950805 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:53.950808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:53.950910 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:53.950940 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:43:53.951315 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:43:53.951325 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 00:43:53.951408 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 00:43:53.951484 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":160,"threads_started":1}
I20260806 00:43:53.973529 32619 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:53.973645 32619 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:53.975663 32619 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56: Sending DeleteTablet for 3 replicas of tablet e146ec596b5841959ffabc6bc2f38e4a
I20260806 00:43:53.976265 539 tablet_service.cc:1514] Processing DeleteTablet for tablet e146ec596b5841959ffabc6bc2f38e4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:53 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:43390
I20260806 00:43:53.976234 354 tablet_service.cc:1514] Processing DeleteTablet for tablet e146ec596b5841959ffabc6bc2f38e4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:53 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:55374
I20260806 00:43:53.976437 989 tablet_replica.cc:330] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: stopping tablet replica
I20260806 00:43:53.976473 987 tablet_replica.cc:330] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: stopping tablet replica
I20260806 00:43:53.976548 989 raft_consensus.cc:2229] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:43:53.976536 757 tablet_service.cc:1514] Processing DeleteTablet for tablet e146ec596b5841959ffabc6bc2f38e4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:53 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53486
I20260806 00:43:53.976593 987 raft_consensus.cc:2229] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:53.976696 987 raft_consensus.cc:2258] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:53.976717 991 tablet_replica.cc:330] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: stopping tablet replica
I20260806 00:43:53.976804 989 raft_consensus.cc:2258] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:53.976854 991 raft_consensus.cc:2229] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:53.976948 991 raft_consensus.cc:2258] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:53.977684 989 ts_tablet_manager.cc:1905] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:53.977703 991 ts_tablet_manager.cc:1905] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:53.977738 987 ts_tablet_manager.cc:1905] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:53.979887 991 ts_tablet_manager.cc:1918] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:43:53.979980 991 log.cc:1196] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/e146ec596b5841959ffabc6bc2f38e4a
I20260806 00:43:53.980055 987 ts_tablet_manager.cc:1918] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:43:53.980055 989 ts_tablet_manager.cc:1918] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:43:53.980137 989 log.cc:1196] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/e146ec596b5841959ffabc6bc2f38e4a
I20260806 00:43:53.980137 987 log.cc:1196] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/e146ec596b5841959ffabc6bc2f38e4a
I20260806 00:43:53.980288 991 ts_tablet_manager.cc:1939] T e146ec596b5841959ffabc6bc2f38e4a P d0575f8ffb4f4e8283d09049cac1f85d: Deleting consensus metadata
I20260806 00:43:53.980403 989 ts_tablet_manager.cc:1939] T e146ec596b5841959ffabc6bc2f38e4a P f805208cd6004d468810764351bc299a: Deleting consensus metadata
I20260806 00:43:53.980414 987 ts_tablet_manager.cc:1939] T e146ec596b5841959ffabc6bc2f38e4a P 65d31a838d804e838a454f579267c0ab: Deleting consensus metadata
I20260806 00:43:53.980681 32619 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 00:43:53.981055 32605 catalog_manager.cc:4841] TS d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131:45611): tablet e146ec596b5841959ffabc6bc2f38e4a (table test-table [id=b7652f7c534346a5b8dc769d896557b0]) successfully deleted
W20260806 00:43:53.981096 32619 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:43:53.981447 32605 catalog_manager.cc:4841] TS 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489): tablet e146ec596b5841959ffabc6bc2f38e4a (table test-table [id=b7652f7c534346a5b8dc769d896557b0]) successfully deleted
I20260806 00:43:53.981979 32602 catalog_manager.cc:4841] TS f805208cd6004d468810764351bc299a (127.29.84.130:38929): tablet e146ec596b5841959ffabc6bc2f38e4a (table test-table [id=b7652f7c534346a5b8dc769d896557b0]) successfully deleted
I20260806 00:43:53.985638 539 tablet_service.cc:1467] Processing CreateTablet for tablet 15627f563055457d806bff1da7684db8 (DEFAULT_TABLE table=test-table [id=62336027ea9c4556b4644682e113b437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:53.985600 354 tablet_service.cc:1467] Processing CreateTablet for tablet 15627f563055457d806bff1da7684db8 (DEFAULT_TABLE table=test-table [id=62336027ea9c4556b4644682e113b437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:53.985941 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15627f563055457d806bff1da7684db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:53.985947 354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15627f563055457d806bff1da7684db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:53.987694 1002 tablet_bootstrap.cc:492] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a: Bootstrap starting.
I20260806 00:43:53.988390 1003 tablet_bootstrap.cc:492] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab: Bootstrap starting.
I20260806 00:43:53.989284 1002 tablet_bootstrap.cc:654] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:53.990012 1003 tablet_bootstrap.cc:654] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:53.990808 1002 tablet_bootstrap.cc:492] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a: No bootstrap required, opened a new log
I20260806 00:43:53.990893 1002 ts_tablet_manager.cc:1397] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260806 00:43:53.991472 757 tablet_service.cc:1467] Processing CreateTablet for tablet 15627f563055457d806bff1da7684db8 (DEFAULT_TABLE table=test-table [id=62336027ea9c4556b4644682e113b437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:43:53.991760 757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15627f563055457d806bff1da7684db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:53.991744 1002 raft_consensus.cc:348] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.991870 1002 raft_consensus.cc:374] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:53.991910 1002 raft_consensus.cc:729] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f805208cd6004d468810764351bc299a, State: Initialized, Role: FOLLOWER
I20260806 00:43:53.992053 1002 consensus_queue.cc:260] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.992424 1002 ts_tablet_manager.cc:1428] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:43:53.994436 1005 tablet_bootstrap.cc:492] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: Bootstrap starting.
I20260806 00:43:53.995397 1003 tablet_bootstrap.cc:492] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab: No bootstrap required, opened a new log
I20260806 00:43:53.995486 1003 ts_tablet_manager.cc:1397] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260806 00:43:53.995522 1005 tablet_bootstrap.cc:654] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:53.996245 1003 raft_consensus.cc:348] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.996379 1003 raft_consensus.cc:374] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:53.996417 1003 raft_consensus.cc:729] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d31a838d804e838a454f579267c0ab, State: Initialized, Role: FOLLOWER
I20260806 00:43:53.996546 1003 consensus_queue.cc:260] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.996615 1005 tablet_bootstrap.cc:492] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: No bootstrap required, opened a new log
I20260806 00:43:53.996690 1005 ts_tablet_manager.cc:1397] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:53.996815 1003 ts_tablet_manager.cc:1428] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:43:53.997114 1005 raft_consensus.cc:348] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.997228 1005 raft_consensus.cc:374] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:53.997269 1005 raft_consensus.cc:729] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0575f8ffb4f4e8283d09049cac1f85d, State: Initialized, Role: FOLLOWER
I20260806 00:43:53.997402 1005 consensus_queue.cc:260] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:53.997680 1005 ts_tablet_manager.cc:1428] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:43:54.076887 1010 raft_consensus.cc:482] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:43:54.077049 1010 raft_consensus.cc:504] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:54.077489 1010 leader_election.cc:288] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489), d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131:45611)
I20260806 00:43:54.078060 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15627f563055457d806bff1da7684db8" candidate_uuid: "f805208cd6004d468810764351bc299a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab" is_pre_election: true
I20260806 00:43:54.078110 783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15627f563055457d806bff1da7684db8" candidate_uuid: "f805208cd6004d468810764351bc299a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" is_pre_election: true
I20260806 00:43:54.078245 783 raft_consensus.cc:2454] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f805208cd6004d468810764351bc299a in term 0.
I20260806 00:43:54.078245 382 raft_consensus.cc:2454] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f805208cd6004d468810764351bc299a in term 0.
I20260806 00:43:54.078555 491 leader_election.cc:302] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65d31a838d804e838a454f579267c0ab, f805208cd6004d468810764351bc299a; no voters:
I20260806 00:43:54.078770 1010 raft_consensus.cc:2781] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:43:54.078855 1010 raft_consensus.cc:482] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:43:54.078908 1010 raft_consensus.cc:3037] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:54.080004 1010 raft_consensus.cc:504] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:54.080328 1010 leader_election.cc:288] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [CANDIDATE]: Term 1 election: Requested vote from peers 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489), d0575f8ffb4f4e8283d09049cac1f85d (127.29.84.131:45611)
I20260806 00:43:54.080683 783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15627f563055457d806bff1da7684db8" candidate_uuid: "f805208cd6004d468810764351bc299a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0575f8ffb4f4e8283d09049cac1f85d"
I20260806 00:43:54.080721 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15627f563055457d806bff1da7684db8" candidate_uuid: "f805208cd6004d468810764351bc299a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d31a838d804e838a454f579267c0ab"
I20260806 00:43:54.080843 382 raft_consensus.cc:3037] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:54.080849 783 raft_consensus.cc:3037] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:54.081916 783 raft_consensus.cc:2454] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f805208cd6004d468810764351bc299a in term 1.
I20260806 00:43:54.082214 491 leader_election.cc:302] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0575f8ffb4f4e8283d09049cac1f85d, f805208cd6004d468810764351bc299a; no voters:
I20260806 00:43:54.082408 1010 raft_consensus.cc:2781] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:54.082535 1010 raft_consensus.cc:686] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 1 LEADER]: Becoming Leader. State: Replica: f805208cd6004d468810764351bc299a, State: Running, Role: LEADER
I20260806 00:43:54.082543 382 raft_consensus.cc:2454] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f805208cd6004d468810764351bc299a in term 1.
I20260806 00:43:54.082672 1010 consensus_queue.cc:237] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } }
I20260806 00:43:54.084748 32621 catalog_manager.cc:5495] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a reported cstate change: term changed from 0 to 1, leader changed from <none> to f805208cd6004d468810764351bc299a (127.29.84.130). New cstate: current_term: 1 leader_uuid: "f805208cd6004d468810764351bc299a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f805208cd6004d468810764351bc299a" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 } health_report { overall_health: UNKNOWN } } }
I20260806 00:43:54.123756 380 raft_consensus.cc:1261] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Refusing update from remote peer f805208cd6004d468810764351bc299a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:54.124104 783 raft_consensus.cc:1261] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Refusing update from remote peer f805208cd6004d468810764351bc299a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:43:54.124121 1010 consensus_queue.cc:1035] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [LEADER]: Connected to new peer: Peer: permanent_uuid: "65d31a838d804e838a454f579267c0ab" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:43:54.124490 1010 consensus_queue.cc:1035] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0575f8ffb4f4e8283d09049cac1f85d" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:43:54.140619 965 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 00:43:55.697031 32619 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:55.697177 32619 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:43:55.699412 32619 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6cbb26b5d422470e8f192115a6fa1c56: Sending DeleteTablet for 3 replicas of tablet 15627f563055457d806bff1da7684db8
I20260806 00:43:55.700009 354 tablet_service.cc:1514] Processing DeleteTablet for tablet 15627f563055457d806bff1da7684db8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:55 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:55374
I20260806 00:43:55.700003 539 tablet_service.cc:1514] Processing DeleteTablet for tablet 15627f563055457d806bff1da7684db8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:55 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:43390
I20260806 00:43:55.700325 1093 tablet_replica.cc:330] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab: stopping tablet replica
I20260806 00:43:55.700485 1093 raft_consensus.cc:2229] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:55.700491 1091 tablet_replica.cc:330] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a: stopping tablet replica
I20260806 00:43:55.700605 1091 raft_consensus.cc:2229] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:43:55.700639 1093 raft_consensus.cc:2258] T 15627f563055457d806bff1da7684db8 P 65d31a838d804e838a454f579267c0ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:55.700724 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 32724
I20260806 00:43:55.705360 757 tablet_service.cc:1514] Processing DeleteTablet for tablet 15627f563055457d806bff1da7684db8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:43:55 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53486
I20260806 00:43:55.707010 1091 raft_consensus.cc:2258] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:55.707800 1095 tablet_replica.cc:330] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: stopping tablet replica
I20260806 00:43:55.707954 1095 raft_consensus.cc:2229] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:43:55.708101 1095 raft_consensus.cc:2258] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:43:55.708796 1095 ts_tablet_manager.cc:1905] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:43:55.711544 1091 ts_tablet_manager.cc:1905] T 15627f563055457d806bff1da7684db8 P f805208cd6004d468810764351bc299a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 00:43:55.714030 32605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:40489 (error 108)
I20260806 00:43:55.714102 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 459
I20260806 00:43:55.719455 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:55.718928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:40489 (local address 127.0.0.1:55378)
0806 00:43:55.719223 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:55.719317 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:40489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
W20260806 00:43:55.719601 32605 catalog_manager.cc:4584] TS 65d31a838d804e838a454f579267c0ab (127.29.84.129:40489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15627f563055457d806bff1da7684db8: Network error: Client connection negotiation failed: client connection to 127.29.84.129:40489: connect: Connection refused (error 111)
I20260806 00:43:55.736040 1095 ts_tablet_manager.cc:1918] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:43:55.736152 1095 log.cc:1196] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/15627f563055457d806bff1da7684db8
I20260806 00:43:55.736488 1095 ts_tablet_manager.cc:1939] T 15627f563055457d806bff1da7684db8 P d0575f8ffb4f4e8283d09049cac1f85d: Deleting consensus metadata
I20260806 00:43:55.737602 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 663
I20260806 00:43:55.744916 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:55.737688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:38929 (local address 127.0.0.1:43408)
0806 00:43:55.744747 (+ 7059us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:55.744799 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.130:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6974}
I20260806 00:43:55.765947 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 32575
2026-08-06T00:43:55Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6603 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:43:57 dist-test-slave-n326 krb5kdc[1104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:43:57 dist-test-slave-n326 krb5kdc[1104](info): set up 2 sockets
Aug 06 00:43:57 dist-test-slave-n326 krb5kdc[1104](info): commencing operation
krb5kdc: starting...
W20260806 00:43:58.443828 30034 mini_kdc.cc:121] Time spent starting KDC: real 2.649s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:43:58 dist-test-slave-n326 krb5kdc[1104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:43:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:43:58.560123 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:46565
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:46565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:58.667454 1123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:58.667661 1123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:58.667697 1123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:58.671252 1123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:43:58.671289 1123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:58.671305 1123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:58.671320 1123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:43:58.671334 1123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:43:58.676044 1123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:46565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:46565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:58.676893 1123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:58.677757 1123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:58.686033 1133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:58.686050 1136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:58.686074 1123 server_base.cc:1034] running on GCE node
W20260806 00:43:58.686087 1132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:58.686866 1123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:58.687872 1123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:58.689062 1123 hybrid_clock.cc:648] HybridClock initialized: now 1785977038689044 us; error 28 us; skew 500 ppm
Aug 06 00:43:58 dist-test-slave-n326 krb5kdc[1104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977038, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:58.691625 1123 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:43:58.692404 1123 webserver.cc:460] Webserver started at http://127.29.84.190:38359/ using document root <none> and password file <none>
I20260806 00:43:58.692915 1123 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:58.693019 1123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:58.693231 1123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:58.695215 1123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "9ff4133d61d24cedb3b6684c38ad4b93"
format_stamp: "Formatted at 2026-08-06 00:43:58 on dist-test-slave-n326"
server_key: "63a651c490b0b1331df7e32be932a606"
server_key_iv: "77f0d221bfb30eef96d1a711eda73df1"
server_key_version: "encryptionkey@0"
I20260806 00:43:58.695765 1123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "9ff4133d61d24cedb3b6684c38ad4b93"
format_stamp: "Formatted at 2026-08-06 00:43:58 on dist-test-slave-n326"
server_key: "63a651c490b0b1331df7e32be932a606"
server_key_iv: "77f0d221bfb30eef96d1a711eda73df1"
server_key_version: "encryptionkey@0"
I20260806 00:43:58.699460 1123 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:43:58.701671 1144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:58.702804 1123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:43:58.702936 1123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "9ff4133d61d24cedb3b6684c38ad4b93"
format_stamp: "Formatted at 2026-08-06 00:43:58 on dist-test-slave-n326"
server_key: "63a651c490b0b1331df7e32be932a606"
server_key_iv: "77f0d221bfb30eef96d1a711eda73df1"
server_key_version: "encryptionkey@0"
I20260806 00:43:58.703050 1123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:58.727484 1123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:58.730098 1123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:58.730289 1123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:58.737407 1123 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:46565
I20260806 00:43:58.737442 1210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:46565 every 8 connection(s)
I20260806 00:43:58.738300 1123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:43:58.740808 1211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:43:58.745321 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 1123
I20260806 00:43:58.745419 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:43:58.745671 30034 external_mini_cluster.cc:1428] Setting key 498c7beeba9a9b1937ddc901c3188c2c
I20260806 00:43:58.746151 1211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93: Bootstrap starting.
I20260806 00:43:58.748543 1211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93: Neither blocks nor log segments found. Creating new log.
I20260806 00:43:58.749259 1211 log.cc:824] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93: Log is configured to *not* fsync() on all Append() calls
I20260806 00:43:58.751389 1211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93: No bootstrap required, opened a new log
Aug 06 00:43:58 dist-test-slave-n326 krb5kdc[1104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:58.755693 1211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff4133d61d24cedb3b6684c38ad4b93" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46565 } }
I20260806 00:43:58.755971 1211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:43:58.756035 1211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff4133d61d24cedb3b6684c38ad4b93, State: Initialized, Role: FOLLOWER
I20260806 00:43:58.756537 1211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff4133d61d24cedb3b6684c38ad4b93" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46565 } }
I20260806 00:43:58.756702 1211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:43:58.756769 1211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:43:58.756866 1211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:43:58.757915 1211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff4133d61d24cedb3b6684c38ad4b93" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46565 } }
I20260806 00:43:58.758291 1211 leader_election.cc:302] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ff4133d61d24cedb3b6684c38ad4b93; no voters:
I20260806 00:43:58.758617 1211 leader_election.cc:288] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:43:58.758724 1219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:43:58.758956 1219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [term 1 LEADER]: Becoming Leader. State: Replica: 9ff4133d61d24cedb3b6684c38ad4b93, State: Running, Role: LEADER
I20260806 00:43:58.759320 1219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff4133d61d24cedb3b6684c38ad4b93" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46565 } }
I20260806 00:43:58.759855 1211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:43:58.760545 1219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ff4133d61d24cedb3b6684c38ad4b93. Latest consensus state: current_term: 1 leader_uuid: "9ff4133d61d24cedb3b6684c38ad4b93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff4133d61d24cedb3b6684c38ad4b93" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46565 } } }
I20260806 00:43:58.760545 1221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ff4133d61d24cedb3b6684c38ad4b93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff4133d61d24cedb3b6684c38ad4b93" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46565 } } }
I20260806 00:43:58.760779 1221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [sys.catalog]: This master's current role is: LEADER
I20260806 00:43:58.760603 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:58.746974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.29.84.190:46565)
0806 00:43:58.747410 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:58.747424 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:58.747457 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:43:58.748048 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:58.748064 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:58.748363 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:58.748682 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:58.749527 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:58.750859 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:58.751828 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:58.752487 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:58.752641 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:58.755306 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:58.755313 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:58.755328 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:58.755357 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:58.757371 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:58.757805 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:58.757813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:58.757818 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:58.757974 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:58.758282 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:58.758285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:58.758287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:58.758808 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:58.759000 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:58.759338 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 00:43:58.759666 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":135,"threads_started":1}
I20260806 00:43:58.761354 1219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93 [sys.catalog]: This master's current role is: LEADER
I20260806 00:43:58.761689 1229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:43:58.765152 1229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:43:58.770136 1229 catalog_manager.cc:1347] Generated new cluster ID: 5110e7669c254be7b21f9ccc811a8b70
I20260806 00:43:58.770195 1229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:43:58.782630 1229 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:43:58.783594 1229 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:43:58.788071 1229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ff4133d61d24cedb3b6684c38ad4b93: Generated new TSK 0
I20260806 00:43:58.788606 1229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:43:58.819905 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:46565
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:58.927153 1245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:58.927368 1245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:58.927420 1245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:58.931052 1245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:58.931164 1245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:58.931219 1245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:43:58.935956 1245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:58.936774 1245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:58.937613 1245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:58.946425 1253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:58.946511 1252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:58.946508 1255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:58.946653 1245 server_base.cc:1034] running on GCE node
I20260806 00:43:58.947129 1245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:58.947695 1245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:58.948876 1245 hybrid_clock.cc:648] HybridClock initialized: now 1785977038948865 us; error 24 us; skew 500 ppm
Aug 06 00:43:58 dist-test-slave-n326 krb5kdc[1104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977038, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:58.951536 1245 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:43:58.952306 1245 webserver.cc:460] Webserver started at http://127.29.84.129:39967/ using document root <none> and password file <none>
I20260806 00:43:58.952819 1245 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:58.952917 1245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:58.953123 1245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:58.954923 1245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "7b3fabd8ce984282b5fecde43f61bf4c"
format_stamp: "Formatted at 2026-08-06 00:43:58 on dist-test-slave-n326"
server_key: "cbf878fd29c68b03321a9bd5007741f3"
server_key_iv: "31873c48e9927b356017babff43ba6dd"
server_key_version: "encryptionkey@0"
I20260806 00:43:58.955502 1245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "7b3fabd8ce984282b5fecde43f61bf4c"
format_stamp: "Formatted at 2026-08-06 00:43:58 on dist-test-slave-n326"
server_key: "cbf878fd29c68b03321a9bd5007741f3"
server_key_iv: "31873c48e9927b356017babff43ba6dd"
server_key_version: "encryptionkey@0"
I20260806 00:43:58.958894 1245 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:43:58.961380 1265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:58.962651 1245 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:43:58.962780 1245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "7b3fabd8ce984282b5fecde43f61bf4c"
format_stamp: "Formatted at 2026-08-06 00:43:58 on dist-test-slave-n326"
server_key: "cbf878fd29c68b03321a9bd5007741f3"
server_key_iv: "31873c48e9927b356017babff43ba6dd"
server_key_version: "encryptionkey@0"
I20260806 00:43:58.962884 1245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:58.994267 1245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:58.996958 1245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:58.997146 1245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:58.997694 1245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:43:58.998426 1245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:58.998498 1245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:58.998564 1245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:58.998610 1245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:59.008236 1245 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:34029
I20260806 00:43:59.008279 1418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:34029 every 8 connection(s)
I20260806 00:43:59.009071 1245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:43:59.015642 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 1245
I20260806 00:43:59.015761 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:43:59.016058 30034 external_mini_cluster.cc:1428] Setting key e1d252d703eca1291830b1ff2a5d6bd9
Aug 06 00:43:59 dist-test-slave-n326 krb5kdc[1104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977038, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:59.022976 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:59.010773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:59491 (local address 127.29.84.190:46565)
0806 00:43:59.010904 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:59.010908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:59.011716 (+ 808us) server_negotiation.cc:407] Connection header received
0806 00:43:59.013037 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:59.013040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:59.013098 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:59.013218 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:59.014353 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.014881 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.016439 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.016766 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.016874 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:59.019464 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:59.019467 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:59.019469 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:59.019491 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:59.020958 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:59.021494 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:59.021498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:59.021499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:59.021576 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:59.021915 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:59.021918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:59.021920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:59.022105 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:59.022214 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:59.022691 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 00:43:59.022811 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:43:59.023684 1422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:59.011158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:46565 (local address 127.29.84.129:59491)
0806 00:43:59.011572 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:59.011612 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:59.012823 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:59.013391 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:59.013398 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:59.013759 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:59.014201 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:59.014214 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.015009 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.015012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:59.016275 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:59.016287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.016916 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.016919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:59.017054 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:59.017671 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:59.017692 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:59.019294 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:59.021081 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:59.021087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:59.021100 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:59.021351 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:59.021672 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:59.021675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:59.021677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:59.021811 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:59.022213 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:59.022219 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:59.022444 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 00:43:59.022633 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 00:43:59.022921 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":117,"threads_started":1}
I20260806 00:43:59.024780 1419 heartbeater.cc:347] Connected to a master server at 127.29.84.190:46565
I20260806 00:43:59.024999 1419 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:59.025431 1419 heartbeater.cc:511] Master 127.29.84.190:46565 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:43:59.026856 1163 ts_manager.cc:194] Registered new tserver with Master: 7b3fabd8ce984282b5fecde43f61bf4c (127.29.84.129:34029)
I20260806 00:43:59.027994 1163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:59491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:43:59.042014 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:46565
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:59.147850 1431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:59.148056 1431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:59.148095 1431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:59.151795 1431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:59.151840 1431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:59.151950 1431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:43:59.157279 1431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:59.158183 1431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:59.159090 1431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:59.168501 1439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:59.168511 1443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:59.168628 1431 server_base.cc:1034] running on GCE node
W20260806 00:43:59.168519 1440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:59.169180 1431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:59.169766 1431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:59.170957 1431 hybrid_clock.cc:648] HybridClock initialized: now 1785977039170927 us; error 42 us; skew 500 ppm
Aug 06 00:43:59 dist-test-slave-n326 krb5kdc[1104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977039, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:59.173614 1431 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:43:59.174459 1431 webserver.cc:460] Webserver started at http://127.29.84.130:40473/ using document root <none> and password file <none>
I20260806 00:43:59.175000 1431 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:59.175117 1431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:59.175340 1431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:59.177291 1431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "537b70e46c274d0da95e937caab54736"
format_stamp: "Formatted at 2026-08-06 00:43:59 on dist-test-slave-n326"
server_key: "bdeaedb3579ebc2cfe823df4cfc711d4"
server_key_iv: "a0f9ec377115bce75a7fae5aa65bd9ce"
server_key_version: "encryptionkey@0"
I20260806 00:43:59.177839 1431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "537b70e46c274d0da95e937caab54736"
format_stamp: "Formatted at 2026-08-06 00:43:59 on dist-test-slave-n326"
server_key: "bdeaedb3579ebc2cfe823df4cfc711d4"
server_key_iv: "a0f9ec377115bce75a7fae5aa65bd9ce"
server_key_version: "encryptionkey@0"
I20260806 00:43:59.181351 1431 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:43:59.183701 1450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:59.184808 1431 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:43:59.184952 1431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "537b70e46c274d0da95e937caab54736"
format_stamp: "Formatted at 2026-08-06 00:43:59 on dist-test-slave-n326"
server_key: "bdeaedb3579ebc2cfe823df4cfc711d4"
server_key_iv: "a0f9ec377115bce75a7fae5aa65bd9ce"
server_key_version: "encryptionkey@0"
I20260806 00:43:59.185060 1431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:59.196592 1431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:59.199280 1431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:59.199476 1431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:59.200019 1431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:43:59.200793 1431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:59.200865 1431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:59.200927 1431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:59.200971 1431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:59.210007 1431 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:42307
I20260806 00:43:59.210047 1613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:42307 every 8 connection(s)
I20260806 00:43:59.210831 1431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:43:59.217415 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 1431
I20260806 00:43:59.217530 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:43:59.217849 30034 external_mini_cluster.cc:1428] Setting key 97c0c7997db49606d4a817dee5ed3bfe
Aug 06 00:43:59 dist-test-slave-n326 krb5kdc[1104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977039, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:59.224946 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:59.212556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:41985 (local address 127.29.84.190:46565)
0806 00:43:59.212715 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:59.212721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:59.213489 (+ 768us) server_negotiation.cc:407] Connection header received
0806 00:43:59.214723 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:59.214726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:59.214771 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:59.214844 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:59.215974 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.216494 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.217825 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.218209 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.218330 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:59.221105 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:59.221108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:59.221110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:59.221132 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:59.222860 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:59.223437 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:59.223440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:59.223442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:59.223524 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:59.223889 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:59.223892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:59.223894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:59.224071 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:59.224181 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:59.224658 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 00:43:59.224779 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:43:59.225677 1618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:59.212914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:46565 (local address 127.29.84.130:41985)
0806 00:43:59.213331 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:59.213372 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:59.214542 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:59.214998 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:59.215005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:59.215415 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:59.215852 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:59.215866 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.216604 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.216607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:59.217706 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:59.217719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.218371 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.218374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:59.218627 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:59.219299 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:59.219336 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:59.220937 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:59.222978 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:59.222984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:59.222998 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:59.223325 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:59.223641 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:59.223644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:59.223646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:59.223783 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:59.224180 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:59.224186 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:59.224414 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 00:43:59.224589 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 00:43:59.224891 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":113,"threads_started":1}
I20260806 00:43:59.226922 1616 heartbeater.cc:347] Connected to a master server at 127.29.84.190:46565
I20260806 00:43:59.227176 1616 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:59.227620 1616 heartbeater.cc:511] Master 127.29.84.190:46565 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:43:59.228619 1163 ts_manager.cc:194] Registered new tserver with Master: 537b70e46c274d0da95e937caab54736 (127.29.84.130:42307)
I20260806 00:43:59.229188 1163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:41985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:43:59.246016 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:46565
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:43:59.358968 1621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:43:59.359237 1621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:43:59.359280 1621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:43:59.363332 1621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:43:59.363379 1621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:43:59.363438 1621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:43:59.368459 1621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:43:59.369338 1621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:43:59.370190 1621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:43:59.379208 1635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:59.379231 1629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:43:59.379258 1630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:43:59.379288 1621 server_base.cc:1034] running on GCE node
I20260806 00:43:59.379916 1621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:43:59.380519 1621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:43:59.381705 1621 hybrid_clock.cc:648] HybridClock initialized: now 1785977039381686 us; error 32 us; skew 500 ppm
Aug 06 00:43:59 dist-test-slave-n326 krb5kdc[1104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977039, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:43:59.384370 1621 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:43:59.385205 1621 webserver.cc:460] Webserver started at http://127.29.84.131:45427/ using document root <none> and password file <none>
I20260806 00:43:59.385732 1621 fs_manager.cc:359] Metadata directory not provided
I20260806 00:43:59.385819 1621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:43:59.386042 1621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:43:59.388036 1621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "c379fb5dfba748668ae0f3ca881d6f60"
format_stamp: "Formatted at 2026-08-06 00:43:59 on dist-test-slave-n326"
server_key: "31fc0e549d2e8d202fcc3451e09eadba"
server_key_iv: "1666ceaf8abb93504517385e0d6a3d0a"
server_key_version: "encryptionkey@0"
I20260806 00:43:59.388614 1621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "c379fb5dfba748668ae0f3ca881d6f60"
format_stamp: "Formatted at 2026-08-06 00:43:59 on dist-test-slave-n326"
server_key: "31fc0e549d2e8d202fcc3451e09eadba"
server_key_iv: "1666ceaf8abb93504517385e0d6a3d0a"
server_key_version: "encryptionkey@0"
I20260806 00:43:59.392189 1621 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:43:59.394470 1649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:59.395692 1621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:43:59.395821 1621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "c379fb5dfba748668ae0f3ca881d6f60"
format_stamp: "Formatted at 2026-08-06 00:43:59 on dist-test-slave-n326"
server_key: "31fc0e549d2e8d202fcc3451e09eadba"
server_key_iv: "1666ceaf8abb93504517385e0d6a3d0a"
server_key_version: "encryptionkey@0"
I20260806 00:43:59.395928 1621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:43:59.417281 1621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:43:59.420360 1621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:43:59.420555 1621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:43:59.421124 1621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:43:59.421932 1621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:43:59.422010 1621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:59.422084 1621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:43:59.422125 1621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:43:59.431021 1621 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:37129
I20260806 00:43:59.431051 1813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:37129 every 8 connection(s)
I20260806 00:43:59.431880 1621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
Aug 06 00:43:59 dist-test-slave-n326 krb5kdc[1104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977039, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:43:59.441787 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 1621
I20260806 00:43:59.441936 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:43:59.442298 30034 external_mini_cluster.cc:1428] Setting key 1bd6247eb704a70a05e61e7bcab48790
I20260806 00:43:59.445048 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:59.433572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:51533 (local address 127.29.84.190:46565)
0806 00:43:59.433707 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:59.433711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:59.434482 (+ 771us) server_negotiation.cc:407] Connection header received
0806 00:43:59.435841 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:59.435845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:59.435922 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:59.436031 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:59.437120 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.437683 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.438768 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.438990 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.439123 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:59.441309 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:43:59.441313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:43:59.441315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:43:59.441345 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:43:59.442968 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:59.443630 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:59.443634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:59.443636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:59.443728 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:43:59.444058 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:43:59.444061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:43:59.444063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:43:59.444280 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:43:59.444382 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:43:59.444846 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 00:43:59.444938 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:43:59.445746 1818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:43:59.433913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:46565 (local address 127.29.84.131:51533)
0806 00:43:59.434347 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 00:43:59.434386 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:43:59.435644 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:43:59.436203 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:43:59.436209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:43:59.436535 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:43:59.436974 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:59.436985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.437812 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.437821 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:59.438645 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:43:59.438654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:43:59.439157 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:43:59.439161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:43:59.439325 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:43:59.439783 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:43:59.439797 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:43:59.441154 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:43:59.443121 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:59.443129 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:59.443142 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:59.443496 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:59.443819 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:43:59.443821 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:43:59.443822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:43:59.443951 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:43:59.444405 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:43:59.444411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:43:59.444643 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 00:43:59.444779 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 00:43:59.445000 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":130,"threads_started":1}
I20260806 00:43:59.446693 1814 heartbeater.cc:347] Connected to a master server at 127.29.84.190:46565
I20260806 00:43:59.446908 1814 heartbeater.cc:464] Registering TS with master...
I20260806 00:43:59.447391 1814 heartbeater.cc:511] Master 127.29.84.190:46565 requested a full tablet report, sending...
I20260806 00:43:59.448295 1163 ts_manager.cc:194] Registered new tserver with Master: c379fb5dfba748668ae0f3ca881d6f60 (127.29.84.131:37129)
I20260806 00:43:59.448798 1163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:51533
I20260806 00:43:59.455897 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 00:43:59.470259 1216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:43:59.468807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.29.84.190:46565)
0806 00:43:59.468959 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:43:59.468963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:43:59.469138 (+ 175us) server_negotiation.cc:407] Connection header received
0806 00:43:59.469304 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:43:59.469311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:43:59.469349 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:43:59.469429 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:43:59.470118 (+ 689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53532: BlockingRecv error: recv got EOF from 127.0.0.1:53532 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:43:59.472082 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 1245
I20260806 00:43:59.481482 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 1431
I20260806 00:43:59.505317 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 1621
I20260806 00:43:59.516479 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 1123
2026-08-06T00:43:59Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3747 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:43:59 dist-test-slave-n326 krb5kdc[1832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:43:59 dist-test-slave-n326 krb5kdc[1832](info): set up 2 sockets
Aug 06 00:43:59 dist-test-slave-n326 krb5kdc[1832](info): commencing operation
krb5kdc: starting...
W20260806 00:44:00.713999 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:00 dist-test-slave-n326 krb5kdc[1832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:44:00.836104 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38775
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:38775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:00.942019 1853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:00.942232 1853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:00.942270 1853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:00.946043 1853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:00.946085 1853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:00.946103 1853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:00.946137 1853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:00.946152 1853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:00.950968 1853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:38775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:00.951901 1853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:00.952773 1853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:00.960608 1860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:00.960644 1853 server_base.cc:1034] running on GCE node
W20260806 00:44:00.960652 1864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:00.960577 1867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:00.961397 1853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:00.962411 1853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:00.963606 1853 hybrid_clock.cc:648] HybridClock initialized: now 1785977040963579 us; error 45 us; skew 500 ppm
Aug 06 00:44:00 dist-test-slave-n326 krb5kdc[1832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977040, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:00.966217 1853 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:44:00.967180 1853 webserver.cc:460] Webserver started at http://127.29.84.190:42207/ using document root <none> and password file <none>
I20260806 00:44:00.967698 1853 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:00.967789 1853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:00.968016 1853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:00.969897 1853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "67271deb03b6436a90d411fcd52d0dcc"
format_stamp: "Formatted at 2026-08-06 00:44:00 on dist-test-slave-n326"
server_key: "00c552fe31aaa0a03a0bb1813fb7eacc"
server_key_iv: "aed24a5ceb51315199474183e7a77729"
server_key_version: "encryptionkey@0"
I20260806 00:44:00.970438 1853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "67271deb03b6436a90d411fcd52d0dcc"
format_stamp: "Formatted at 2026-08-06 00:44:00 on dist-test-slave-n326"
server_key: "00c552fe31aaa0a03a0bb1813fb7eacc"
server_key_iv: "aed24a5ceb51315199474183e7a77729"
server_key_version: "encryptionkey@0"
I20260806 00:44:00.973978 1853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 00:44:00.976262 1878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:00.977324 1853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:00.977475 1853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "67271deb03b6436a90d411fcd52d0dcc"
format_stamp: "Formatted at 2026-08-06 00:44:00 on dist-test-slave-n326"
server_key: "00c552fe31aaa0a03a0bb1813fb7eacc"
server_key_iv: "aed24a5ceb51315199474183e7a77729"
server_key_version: "encryptionkey@0"
I20260806 00:44:00.977583 1853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:01.012910 1853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:01.015697 1853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:01.015893 1853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:01.022997 1853 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:38775
I20260806 00:44:01.023303 1953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:38775 every 8 connection(s)
I20260806 00:44:01.023959 1853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:01.026486 1954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:01.031793 1954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc: Bootstrap starting.
I20260806 00:44:01.031975 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 1853
I20260806 00:44:01.032105 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:01.032354 30034 external_mini_cluster.cc:1428] Setting key 2aef78d41b808a8a10219bab159dc0e6
I20260806 00:44:01.034469 1954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:01.035204 1954 log.cc:824] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:01.037226 1954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc: No bootstrap required, opened a new log
Aug 06 00:44:01 dist-test-slave-n326 krb5kdc[1832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:01.041563 1954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67271deb03b6436a90d411fcd52d0dcc" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38775 } }
I20260806 00:44:01.041846 1954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:01.041929 1954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67271deb03b6436a90d411fcd52d0dcc, State: Initialized, Role: FOLLOWER
I20260806 00:44:01.042364 1954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67271deb03b6436a90d411fcd52d0dcc" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38775 } }
I20260806 00:44:01.042512 1954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:01.042583 1954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:01.042699 1954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:01.043557 1954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67271deb03b6436a90d411fcd52d0dcc" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38775 } }
I20260806 00:44:01.043900 1954 leader_election.cc:302] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67271deb03b6436a90d411fcd52d0dcc; no voters:
I20260806 00:44:01.044236 1954 leader_election.cc:288] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:01.044279 1964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:01.044481 1964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [term 1 LEADER]: Becoming Leader. State: Replica: 67271deb03b6436a90d411fcd52d0dcc, State: Running, Role: LEADER
I20260806 00:44:01.044761 1964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67271deb03b6436a90d411fcd52d0dcc" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38775 } }
I20260806 00:44:01.045403 1954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:01.045972 1966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67271deb03b6436a90d411fcd52d0dcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67271deb03b6436a90d411fcd52d0dcc" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38775 } } }
I20260806 00:44:01.046106 1966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:01.046370 1965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67271deb03b6436a90d411fcd52d0dcc. Latest consensus state: current_term: 1 leader_uuid: "67271deb03b6436a90d411fcd52d0dcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67271deb03b6436a90d411fcd52d0dcc" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38775 } } }
I20260806 00:44:01.046516 1965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:01.046618 1969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:01.048348 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:01.033966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46672 (local address 127.29.84.190:38775)
0806 00:44:01.034364 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:01.034377 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:01.034408 (+ 31us) server_negotiation.cc:407] Connection header received
0806 00:44:01.035012 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:01.035027 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:01.035445 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:01.035779 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:01.036644 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:01.037683 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:01.038868 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:01.039358 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:01.039491 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:01.042256 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:01.042261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:01.042272 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:01.042296 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:01.044120 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:01.044812 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:01.044821 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:01.044828 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:01.045039 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:01.045373 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:01.045376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:01.045378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:01.045899 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:01.046802 (+ 903us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:01.046979 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 00:44:01.047364 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":128,"threads_started":1}
I20260806 00:44:01.049827 1969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:01.051426 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:44:01.057696 1969 catalog_manager.cc:1347] Generated new cluster ID: 0fbc8f9dbfb4421287b77204783f68e0
I20260806 00:44:01.057780 1969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:01.077230 1969 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:01.078758 1969 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:01.085409 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:01.078064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.29.84.190:38775)
0806 00:44:01.078163 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:01.078168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:01.078779 (+ 611us) server_negotiation.cc:407] Connection header received
0806 00:44:01.078952 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:01.078956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:01.079001 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:01.079050 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:01.079760 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:01.080518 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:01.081787 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:01.082138 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:01.082209 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:01.084083 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:01.084090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:01.084096 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:01.084924 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:01.084957 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:01.084977 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:01.085058 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:01.085206 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 00:44:01.085265 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 00:44:01.091107 1969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67271deb03b6436a90d411fcd52d0dcc: Generated new TSK 0
I20260806 00:44:01.092283 1969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:44:01.100100 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 1853
2026-08-06T00:44:01Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1588 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:01 dist-test-slave-n326 krb5kdc[2000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:01 dist-test-slave-n326 krb5kdc[2000](info): set up 2 sockets
Aug 06 00:44:01 dist-test-slave-n326 krb5kdc[2000](info): commencing operation
krb5kdc: starting...
W20260806 00:44:02.338007 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:02 dist-test-slave-n326 krb5kdc[2000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:44:02.462294 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:34423
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:34423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:02.568994 2018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:02.569212 2018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:02.569250 2018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:02.573056 2018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:02.573103 2018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:02.573122 2018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:02.573139 2018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:02.573154 2018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:02.578245 2018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:34423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:34423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:02.579243 2018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:02.580264 2018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:02.588085 2031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:02.588061 2026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:02.588086 2029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:02.588356 2018 server_base.cc:1034] running on GCE node
I20260806 00:44:02.588960 2018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:02.590196 2018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:02.591394 2018 hybrid_clock.cc:648] HybridClock initialized: now 1785977042591379 us; error 49 us; skew 500 ppm
Aug 06 00:44:02 dist-test-slave-n326 krb5kdc[2000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977042, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:02.600236 2018 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:44:02.601048 2018 webserver.cc:460] Webserver started at http://127.29.84.190:35993/ using document root <none> and password file <none>
I20260806 00:44:02.601583 2018 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:02.601675 2018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:02.601912 2018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:02.603886 2018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "922d488de0fe4fe1aff3df1a74a10c05"
format_stamp: "Formatted at 2026-08-06 00:44:02 on dist-test-slave-n326"
server_key: "34095461d59de463d9e4364cc14c7887"
server_key_iv: "e33d7cafdfb99c8d177e374e4c45f0b5"
server_key_version: "encryptionkey@0"
I20260806 00:44:02.604442 2018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "922d488de0fe4fe1aff3df1a74a10c05"
format_stamp: "Formatted at 2026-08-06 00:44:02 on dist-test-slave-n326"
server_key: "34095461d59de463d9e4364cc14c7887"
server_key_iv: "e33d7cafdfb99c8d177e374e4c45f0b5"
server_key_version: "encryptionkey@0"
I20260806 00:44:02.608016 2018 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:02.610333 2039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:02.611593 2018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:02.611732 2018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "922d488de0fe4fe1aff3df1a74a10c05"
format_stamp: "Formatted at 2026-08-06 00:44:02 on dist-test-slave-n326"
server_key: "34095461d59de463d9e4364cc14c7887"
server_key_iv: "e33d7cafdfb99c8d177e374e4c45f0b5"
server_key_version: "encryptionkey@0"
I20260806 00:44:02.611840 2018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:02.631166 2018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:02.634176 2018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:02.634370 2018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:02.640863 2018 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:34423
I20260806 00:44:02.641146 2115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:34423 every 8 connection(s)
I20260806 00:44:02.641769 2018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:02.644316 2117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:02.647694 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 2018
I20260806 00:44:02.647854 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:02.648128 30034 external_mini_cluster.cc:1428] Setting key 1e237e4bffb7ce49f3ce1c66eb6652ad
I20260806 00:44:02.649638 2117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05: Bootstrap starting.
I20260806 00:44:02.652026 2117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:02.652737 2117 log.cc:824] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:02.654716 2117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05: No bootstrap required, opened a new log
Aug 06 00:44:02 dist-test-slave-n326 krb5kdc[2000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:02.659205 2117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d488de0fe4fe1aff3df1a74a10c05" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 34423 } }
I20260806 00:44:02.659472 2117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:02.659533 2117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922d488de0fe4fe1aff3df1a74a10c05, State: Initialized, Role: FOLLOWER
I20260806 00:44:02.659979 2117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d488de0fe4fe1aff3df1a74a10c05" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 34423 } }
I20260806 00:44:02.660149 2117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:02.660221 2117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:02.660322 2117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:02.661123 2117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d488de0fe4fe1aff3df1a74a10c05" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 34423 } }
I20260806 00:44:02.661530 2117 leader_election.cc:302] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 922d488de0fe4fe1aff3df1a74a10c05; no voters:
I20260806 00:44:02.661895 2117 leader_election.cc:288] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:02.661959 2127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:02.662143 2127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [term 1 LEADER]: Becoming Leader. State: Replica: 922d488de0fe4fe1aff3df1a74a10c05, State: Running, Role: LEADER
I20260806 00:44:02.662439 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:02.649332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52086 (local address 127.29.84.190:34423)
0806 00:44:02.649732 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:02.649747 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:02.649806 (+ 59us) server_negotiation.cc:407] Connection header received
0806 00:44:02.650393 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:02.650409 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:02.650699 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:02.650969 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:02.651913 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:02.652928 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:02.654036 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:02.654476 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:02.654599 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:02.657298 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:02.657306 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:02.657328 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:02.657357 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:02.659318 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:02.659866 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:02.659871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:02.659877 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:02.660015 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:02.660351 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:02.660355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:02.660357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:02.660751 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:02.660972 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:02.661293 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 00:44:02.661611 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":120,"threads_started":1}
I20260806 00:44:02.662478 2127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d488de0fe4fe1aff3df1a74a10c05" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 34423 } }
I20260806 00:44:02.663105 2117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:02.664278 2129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 922d488de0fe4fe1aff3df1a74a10c05. Latest consensus state: current_term: 1 leader_uuid: "922d488de0fe4fe1aff3df1a74a10c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d488de0fe4fe1aff3df1a74a10c05" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 34423 } } }
I20260806 00:44:02.664433 2129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:02.664628 2130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "922d488de0fe4fe1aff3df1a74a10c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d488de0fe4fe1aff3df1a74a10c05" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 34423 } } }
I20260806 00:44:02.664932 2130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:02.665113 2139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:02.668279 2139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:02.673359 2139 catalog_manager.cc:1347] Generated new cluster ID: f1c726f1b8d84618838208aebbd8f987
I20260806 00:44:02.673467 2139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:02.682897 2139 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:02.683750 2139 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:02.688088 2139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 922d488de0fe4fe1aff3df1a74a10c05: Generated new TSK 0
I20260806 00:44:02.688647 2139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:44:02.716372 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:34423
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:02.825044 2153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:02.825268 2153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:02.825309 2153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:02.829349 2153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:02.829403 2153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:02.829456 2153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:02.834419 2153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:34423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:02.835330 2153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:02.836267 2153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:02.845191 2160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:02.845191 2162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:02.845235 2159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:02.845466 2153 server_base.cc:1034] running on GCE node
I20260806 00:44:02.845877 2153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:02.846482 2153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:02.847647 2153 hybrid_clock.cc:648] HybridClock initialized: now 1785977042847627 us; error 31 us; skew 500 ppm
Aug 06 00:44:02 dist-test-slave-n326 krb5kdc[2000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977042, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:02.850363 2153 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:44:02.851272 2153 webserver.cc:460] Webserver started at http://127.29.84.129:33917/ using document root <none> and password file <none>
I20260806 00:44:02.851751 2153 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:02.851805 2153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:02.851954 2153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:02.853703 2153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "dd05770d370949779d5a8adc73c22531"
format_stamp: "Formatted at 2026-08-06 00:44:02 on dist-test-slave-n326"
server_key: "31137ee48556aa86c8f618f6160ab2ba"
server_key_iv: "51cdd837f6b237437e821919eb1e50e8"
server_key_version: "encryptionkey@0"
I20260806 00:44:02.854173 2153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "dd05770d370949779d5a8adc73c22531"
format_stamp: "Formatted at 2026-08-06 00:44:02 on dist-test-slave-n326"
server_key: "31137ee48556aa86c8f618f6160ab2ba"
server_key_iv: "51cdd837f6b237437e821919eb1e50e8"
server_key_version: "encryptionkey@0"
I20260806 00:44:02.857611 2153 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 00:44:02.859853 2171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:02.861078 2153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:44:02.861205 2153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "dd05770d370949779d5a8adc73c22531"
format_stamp: "Formatted at 2026-08-06 00:44:02 on dist-test-slave-n326"
server_key: "31137ee48556aa86c8f618f6160ab2ba"
server_key_iv: "51cdd837f6b237437e821919eb1e50e8"
server_key_version: "encryptionkey@0"
I20260806 00:44:02.861310 2153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:02.872571 2153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:02.875610 2153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:02.875798 2153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:02.876315 2153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:02.877044 2153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:02.877123 2153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:02.877187 2153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:02.877235 2153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:02.886236 2153 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:38451
I20260806 00:44:02.886291 2329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:38451 every 8 connection(s)
I20260806 00:44:02.887053 2153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:02.892004 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 2153
I20260806 00:44:02.892153 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:02.892431 30034 external_mini_cluster.cc:1428] Setting key 1b3954ceaf7c80ace2dc32dc3c209890
Aug 06 00:44:02 dist-test-slave-n326 krb5kdc[2000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977042, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:02.900823 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:02.888670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:49351 (local address 127.29.84.190:34423)
0806 00:44:02.888793 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:02.888797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:02.889511 (+ 714us) server_negotiation.cc:407] Connection header received
0806 00:44:02.890767 (+ 1256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:02.890770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:02.890826 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:02.890911 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:02.892218 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:02.892812 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:02.894232 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:02.894639 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:02.894770 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:02.897112 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:02.897122 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:02.897125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:02.897149 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:02.898609 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:02.899167 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:02.899170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:02.899172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:02.899254 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:02.899627 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:02.899631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:02.899636 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:02.899803 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:02.899913 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:02.900478 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 00:44:02.900612 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:44:02.901479 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:02.888966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:34423 (local address 127.29.84.129:49351)
0806 00:44:02.889373 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:02.889410 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:02.890568 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:02.891051 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:02.891057 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:02.891393 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:02.891871 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:02.891891 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:02.892968 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:02.892971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:02.894091 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:02.894102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:02.894796 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:02.894799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:02.894975 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:02.895533 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:02.895547 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:02.896956 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:02.898735 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:02.898741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:02.898754 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:02.899012 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:02.899376 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:02.899378 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:02.899380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:02.899513 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:02.899912 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:02.899918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:02.900191 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 00:44:02.900368 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 00:44:02.900644 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":119,"threads_started":1}
I20260806 00:44:02.902604 2331 heartbeater.cc:347] Connected to a master server at 127.29.84.190:34423
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:44:02.902829 2331 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:02.903321 2331 heartbeater.cc:511] Master 127.29.84.190:34423 requested a full tablet report, sending...
I20260806 00:44:02.904711 2066 ts_manager.cc:194] Registered new tserver with Master: dd05770d370949779d5a8adc73c22531 (127.29.84.129:38451)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
I20260806 00:44:02.905973 2066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:49351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:44:02.922227 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:34423
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:03.028700 2343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:03.028913 2343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:03.028951 2343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:03.032853 2343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:03.032900 2343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:03.032953 2343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:03.037734 2343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:34423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:03.038553 2343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:03.039482 2343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:03.048312 2350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:03.048372 2356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:03.048367 2354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:03.048681 2343 server_base.cc:1034] running on GCE node
I20260806 00:44:03.049049 2343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:03.049636 2343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:03.050825 2343 hybrid_clock.cc:648] HybridClock initialized: now 1785977043050816 us; error 28 us; skew 500 ppm
Aug 06 00:44:03 dist-test-slave-n326 krb5kdc[2000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977043, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:03.053464 2343 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:44:03.054219 2343 webserver.cc:460] Webserver started at http://127.29.84.130:33889/ using document root <none> and password file <none>
I20260806 00:44:03.054728 2343 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:03.054785 2343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:03.054939 2343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:03.056782 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "fd153e742da347c39bd4283ad0927538"
format_stamp: "Formatted at 2026-08-06 00:44:03 on dist-test-slave-n326"
server_key: "760d5fa8e16b4fc3833c9f28f3ba9ac6"
server_key_iv: "e0f66d6e39c0c5d138ba61556cdd0e3a"
server_key_version: "encryptionkey@0"
I20260806 00:44:03.057329 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "fd153e742da347c39bd4283ad0927538"
format_stamp: "Formatted at 2026-08-06 00:44:03 on dist-test-slave-n326"
server_key: "760d5fa8e16b4fc3833c9f28f3ba9ac6"
server_key_iv: "e0f66d6e39c0c5d138ba61556cdd0e3a"
server_key_version: "encryptionkey@0"
I20260806 00:44:03.060778 2343 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:44:03.063612 2364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:03.065230 2343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:44:03.065361 2343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "fd153e742da347c39bd4283ad0927538"
format_stamp: "Formatted at 2026-08-06 00:44:03 on dist-test-slave-n326"
server_key: "760d5fa8e16b4fc3833c9f28f3ba9ac6"
server_key_iv: "e0f66d6e39c0c5d138ba61556cdd0e3a"
server_key_version: "encryptionkey@0"
I20260806 00:44:03.065488 2343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:03.078097 2343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:03.081103 2343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:03.081295 2343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:03.081835 2343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:03.082579 2343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:03.082659 2343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:03.082724 2343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:03.082777 2343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:03.091444 2343 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:43481
I20260806 00:44:03.091472 2526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:43481 every 8 connection(s)
I20260806 00:44:03.092211 2343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:03.097750 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 2343
I20260806 00:44:03.097878 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:03.098172 30034 external_mini_cluster.cc:1428] Setting key 5c277582cb4165e9a916b502d990b0ec
Aug 06 00:44:03 dist-test-slave-n326 krb5kdc[2000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977043, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:03.106107 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:03.093879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:43893 (local address 127.29.84.190:34423)
0806 00:44:03.094010 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:03.094013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:03.094714 (+ 701us) server_negotiation.cc:407] Connection header received
0806 00:44:03.096109 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:03.096112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:03.096189 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:03.096292 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:03.097352 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.097969 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.099209 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.099449 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.099567 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:03.102110 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:03.102116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:03.102121 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:03.102157 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:03.103966 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:03.104457 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:03.104464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:03.104469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:03.104564 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:03.104939 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:03.104944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:03.104948 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:03.105173 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:03.105286 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:03.105713 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 00:44:03.105853 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:44:03.106626 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:03.094130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:34423 (local address 127.29.84.130:43893)
0806 00:44:03.094568 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:03.094607 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:03.095910 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:03.096463 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:03.096469 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:03.096784 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:03.097195 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:03.097207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.098124 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.098127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:03.099028 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:03.099039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.099601 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.099606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:03.099776 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:03.100321 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:03.100337 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:03.101944 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:03.104093 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:03.104099 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:03.104109 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:03.104350 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:03.104682 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:03.104685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:03.104689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:03.104832 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:03.105282 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:03.105287 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:03.105475 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 00:44:03.105625 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 00:44:03.105840 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":123,"threads_started":1}
I20260806 00:44:03.107723 2527 heartbeater.cc:347] Connected to a master server at 127.29.84.190:34423
I20260806 00:44:03.107980 2527 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:03.108529 2527 heartbeater.cc:511] Master 127.29.84.190:34423 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:44:03.109639 2066 ts_manager.cc:194] Registered new tserver with Master: fd153e742da347c39bd4283ad0927538 (127.29.84.130:43481)
I20260806 00:44:03.110212 2066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:43893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:44:03.126029 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:34423
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:03.232607 2535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:03.232836 2535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:03.232893 2535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:03.236644 2535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:03.236690 2535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:03.236785 2535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:03.241550 2535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:34423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:03.242434 2535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:03.243347 2535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:03.252036 2541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:03.252055 2546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:03.252038 2542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:03.252305 2535 server_base.cc:1034] running on GCE node
I20260806 00:44:03.252780 2535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:03.253350 2535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:03.254554 2535 hybrid_clock.cc:648] HybridClock initialized: now 1785977043254522 us; error 44 us; skew 500 ppm
Aug 06 00:44:03 dist-test-slave-n326 krb5kdc[2000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977043, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:03.257207 2535 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:44:03.258108 2535 webserver.cc:460] Webserver started at http://127.29.84.131:43771/ using document root <none> and password file <none>
I20260806 00:44:03.258662 2535 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:03.258760 2535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:03.259002 2535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:03.261032 2535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "acb92c2defb44902862365129b690f98"
format_stamp: "Formatted at 2026-08-06 00:44:03 on dist-test-slave-n326"
server_key: "4f51d57e5e37f3ad6458c015b8d1dbe7"
server_key_iv: "a1000fe1e8ae10b8e01678d0d4ab4fb2"
server_key_version: "encryptionkey@0"
I20260806 00:44:03.261713 2535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "acb92c2defb44902862365129b690f98"
format_stamp: "Formatted at 2026-08-06 00:44:03 on dist-test-slave-n326"
server_key: "4f51d57e5e37f3ad6458c015b8d1dbe7"
server_key_iv: "a1000fe1e8ae10b8e01678d0d4ab4fb2"
server_key_version: "encryptionkey@0"
I20260806 00:44:03.265815 2535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 00:44:03.268265 2558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:03.269522 2535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:03.269656 2535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "acb92c2defb44902862365129b690f98"
format_stamp: "Formatted at 2026-08-06 00:44:03 on dist-test-slave-n326"
server_key: "4f51d57e5e37f3ad6458c015b8d1dbe7"
server_key_iv: "a1000fe1e8ae10b8e01678d0d4ab4fb2"
server_key_version: "encryptionkey@0"
I20260806 00:44:03.269770 2535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:03.289551 2535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:03.292341 2535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:03.292541 2535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:03.293130 2535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:03.293957 2535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:03.294039 2535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:03.294117 2535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:03.294168 2535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:03.303809 2535 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:45215
I20260806 00:44:03.303833 2721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:45215 every 8 connection(s)
I20260806 00:44:03.304716 2535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:03.312130 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 2535
I20260806 00:44:03.312260 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:03.312541 30034 external_mini_cluster.cc:1428] Setting key 657bff54741dd9874e72ea3f92fbf1cd
Aug 06 00:44:03 dist-test-slave-n326 krb5kdc[2000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977043, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:03.318347 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:03.306487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:38629 (local address 127.29.84.190:34423)
0806 00:44:03.306668 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:03.306673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:03.307553 (+ 880us) server_negotiation.cc:407] Connection header received
0806 00:44:03.308796 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:03.308798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:03.308847 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:03.308929 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:03.310002 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.310577 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.311758 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.312016 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.312148 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:03.315051 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:03.315055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:03.315057 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:03.315139 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:03.316625 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:03.317079 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:03.317082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:03.317083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:03.317159 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:03.317456 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:03.317458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:03.317459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:03.317627 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:03.317709 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:03.318117 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 00:44:03.318215 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 00:44:03.319036 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:03.306931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:34423 (local address 127.29.84.131:38629)
0806 00:44:03.307404 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:03.307445 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:03.308600 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:03.309075 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:03.309082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:03.309434 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:03.309863 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:03.309875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.310686 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.310689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:03.311642 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:03.311652 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.312190 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.312195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:03.312448 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:03.313055 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:03.313079 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:03.314909 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:03.316732 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:03.316736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:03.316746 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:03.316990 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:03.317248 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:03.317250 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:03.317252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:03.317373 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:03.317717 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:03.317721 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:03.317919 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 00:44:03.318061 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 00:44:03.318305 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":158,"threads_started":1}
I20260806 00:44:03.320153 2723 heartbeater.cc:347] Connected to a master server at 127.29.84.190:34423
I20260806 00:44:03.320402 2723 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:03.320853 2723 heartbeater.cc:511] Master 127.29.84.190:34423 requested a full tablet report, sending...
I20260806 00:44:03.321839 2066 ts_manager.cc:194] Registered new tserver with Master: acb92c2defb44902862365129b690f98 (127.29.84.131:45215)
I20260806 00:44:03.322436 2066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:38629
I20260806 00:44:03.326377 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:44:03 dist-test-slave-n326 krb5kdc[2000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977043, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 00:44:03 dist-test-slave-n326 krb5kdc[2000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977043, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:03.352042 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:03.343713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.29.84.190:34423)
0806 00:44:03.343872 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:03.343878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:03.343990 (+ 112us) server_negotiation.cc:407] Connection header received
0806 00:44:03.344087 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:03.344089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:03.344133 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:03.344196 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:03.344907 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.345747 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.346742 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:03.346946 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:03.347022 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:03.349329 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:03.349333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:03.349335 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:03.349357 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:03.350556 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:03.350931 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:03.350934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:03.350935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:03.351008 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:03.351303 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:03.351305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:03.351306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:03.351508 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:03.351642 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:03.351828 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 00:44:03.351928 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 00:44:03.352403 2066 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52092
I20260806 00:44:03.353310 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 2153
I20260806 00:44:03.376976 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 2343
I20260806 00:44:03.392078 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 2535
I20260806 00:44:03.402621 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 2018
2026-08-06T00:44:03Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2300 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:03 dist-test-slave-n326 krb5kdc[2739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:03 dist-test-slave-n326 krb5kdc[2739](info): set up 2 sockets
Aug 06 00:44:03 dist-test-slave-n326 krb5kdc[2739](info): commencing operation
krb5kdc: starting...
W20260806 00:44:04.614575 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:04 dist-test-slave-n326 krb5kdc[2739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:44:04.733870 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40753
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:40753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:04.839977 2757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:04.840230 2757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:04.840296 2757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:04.844159 2757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:04.844233 2757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:04.844269 2757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:04.844355 2757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:04.844398 2757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:04.849421 2757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:40753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:04.850330 2757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:04.851320 2757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:04.859421 2766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:04.859398 2767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:04.859433 2769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:04.859714 2757 server_base.cc:1034] running on GCE node
I20260806 00:44:04.860239 2757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:04.861181 2757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:04.862340 2757 hybrid_clock.cc:648] HybridClock initialized: now 1785977044862321 us; error 29 us; skew 500 ppm
Aug 06 00:44:04 dist-test-slave-n326 krb5kdc[2739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977044, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:04.864990 2757 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:44:04.865787 2757 webserver.cc:460] Webserver started at http://127.29.84.190:39453/ using document root <none> and password file <none>
I20260806 00:44:04.866259 2757 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:04.866315 2757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:04.866469 2757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:04.868340 2757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "dd9c101d17954704af7dc2d4d74d9321"
format_stamp: "Formatted at 2026-08-06 00:44:04 on dist-test-slave-n326"
server_key: "67bdc7e741d103e7889205288457405f"
server_key_iv: "8ee25007c4defbf347fea4e77bc14f60"
server_key_version: "encryptionkey@0"
I20260806 00:44:04.868827 2757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "dd9c101d17954704af7dc2d4d74d9321"
format_stamp: "Formatted at 2026-08-06 00:44:04 on dist-test-slave-n326"
server_key: "67bdc7e741d103e7889205288457405f"
server_key_iv: "8ee25007c4defbf347fea4e77bc14f60"
server_key_version: "encryptionkey@0"
I20260806 00:44:04.872359 2757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:04.874632 2778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:04.875833 2757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:44:04.875970 2757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "dd9c101d17954704af7dc2d4d74d9321"
format_stamp: "Formatted at 2026-08-06 00:44:04 on dist-test-slave-n326"
server_key: "67bdc7e741d103e7889205288457405f"
server_key_iv: "8ee25007c4defbf347fea4e77bc14f60"
server_key_version: "encryptionkey@0"
I20260806 00:44:04.876076 2757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:04.885351 2757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:04.887895 2757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:04.888095 2757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:04.894651 2757 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:40753
I20260806 00:44:04.894663 2855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:40753 every 8 connection(s)
I20260806 00:44:04.895582 2757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:04.898026 2857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:04.899358 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 2757
I20260806 00:44:04.899489 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:04.899761 30034 external_mini_cluster.cc:1428] Setting key 4d97edcd6bfb29cda2b82f02ae7d6a75
I20260806 00:44:04.903923 2857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321: Bootstrap starting.
I20260806 00:44:04.906286 2857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:04.907208 2857 log.cc:824] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:44:04 dist-test-slave-n326 krb5kdc[2739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:04.909005 2857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321: No bootstrap required, opened a new log
I20260806 00:44:04.912477 2857 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9c101d17954704af7dc2d4d74d9321" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40753 } }
I20260806 00:44:04.912746 2857 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:04.912842 2857 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9c101d17954704af7dc2d4d74d9321, State: Initialized, Role: FOLLOWER
I20260806 00:44:04.913228 2857 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9c101d17954704af7dc2d4d74d9321" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40753 } }
I20260806 00:44:04.913370 2857 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:04.913453 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:04.900952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.29.84.190:40753)
0806 00:44:04.901340 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:04.901356 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:04.901423 (+ 67us) server_negotiation.cc:407] Connection header received
0806 00:44:04.902012 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:04.902027 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:04.902307 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:04.902579 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:04.903572 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:04.904542 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:04.905721 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:04.906151 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:04.906262 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:04.908898 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:04.908903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:04.908914 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:04.908939 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:04.910576 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:04.911023 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:04.911027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:04.911031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:04.911196 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:04.911486 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:04.911490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:04.911492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:04.911838 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:04.912062 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:04.912342 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 00:44:04.912600 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":127,"threads_started":1}
I20260806 00:44:04.913455 2857 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:04.913652 2857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:04.914485 2857 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9c101d17954704af7dc2d4d74d9321" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40753 } }
I20260806 00:44:04.914827 2857 leader_election.cc:302] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd9c101d17954704af7dc2d4d74d9321; no voters:
I20260806 00:44:04.915153 2857 leader_election.cc:288] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:04.915264 2865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:04.915506 2865 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [term 1 LEADER]: Becoming Leader. State: Replica: dd9c101d17954704af7dc2d4d74d9321, State: Running, Role: LEADER
I20260806 00:44:04.915823 2865 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9c101d17954704af7dc2d4d74d9321" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40753 } }
I20260806 00:44:04.916253 2857 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:04.917465 2867 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd9c101d17954704af7dc2d4d74d9321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9c101d17954704af7dc2d4d74d9321" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40753 } } }
I20260806 00:44:04.917448 2868 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd9c101d17954704af7dc2d4d74d9321. Latest consensus state: current_term: 1 leader_uuid: "dd9c101d17954704af7dc2d4d74d9321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9c101d17954704af7dc2d4d74d9321" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40753 } } }
I20260806 00:44:04.917594 2867 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:04.917594 2868 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:04.918047 2876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:04.920989 2876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:04.925659 2876 catalog_manager.cc:1347] Generated new cluster ID: 6d963f78de7247c496b2cfdf9d0e61f3
I20260806 00:44:04.925719 2876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:04.937505 2876 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:04.938315 2876 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:04.946786 2876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd9c101d17954704af7dc2d4d74d9321: Generated new TSK 0
I20260806 00:44:04.947433 2876 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:44:04.978114 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40753
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:05.085752 2893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:05.085964 2893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:05.086004 2893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:05.089737 2893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:05.089782 2893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:05.089830 2893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:05.094349 2893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:05.095227 2893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:05.096081 2893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:05.105188 2906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:05.105163 2900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:05.105237 2899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:05.105320 2893 server_base.cc:1034] running on GCE node
I20260806 00:44:05.105799 2893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:05.106388 2893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:05.107558 2893 hybrid_clock.cc:648] HybridClock initialized: now 1785977045107527 us; error 43 us; skew 500 ppm
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[2739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977045, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:05.110280 2893 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:44:05.111024 2893 webserver.cc:460] Webserver started at http://127.29.84.129:35353/ using document root <none> and password file <none>
I20260806 00:44:05.111554 2893 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:05.111621 2893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:05.111840 2893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:05.113699 2893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "e86ee1b10ec24869a066067cf8ea63c7"
format_stamp: "Formatted at 2026-08-06 00:44:05 on dist-test-slave-n326"
server_key: "568e8621ca516850525319540cdacd61"
server_key_iv: "66d8eb53faaace43ae9081f450a6b8b4"
server_key_version: "encryptionkey@0"
I20260806 00:44:05.114223 2893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "e86ee1b10ec24869a066067cf8ea63c7"
format_stamp: "Formatted at 2026-08-06 00:44:05 on dist-test-slave-n326"
server_key: "568e8621ca516850525319540cdacd61"
server_key_iv: "66d8eb53faaace43ae9081f450a6b8b4"
server_key_version: "encryptionkey@0"
I20260806 00:44:05.118009 2893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:44:05.120280 2915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:05.121364 2893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:05.121532 2893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "e86ee1b10ec24869a066067cf8ea63c7"
format_stamp: "Formatted at 2026-08-06 00:44:05 on dist-test-slave-n326"
server_key: "568e8621ca516850525319540cdacd61"
server_key_iv: "66d8eb53faaace43ae9081f450a6b8b4"
server_key_version: "encryptionkey@0"
I20260806 00:44:05.121646 2893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:05.141014 2893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:05.144110 2893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:05.144312 2893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:05.144838 2893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:05.145568 2893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:05.145648 2893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:05.145720 2893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:05.145764 2893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:05.154515 2893 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:35543
I20260806 00:44:05.154546 3085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:35543 every 8 connection(s)
I20260806 00:44:05.155319 2893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:05.163733 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 2893
I20260806 00:44:05.163861 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[2739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977045, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:05.164188 30034 external_mini_cluster.cc:1428] Setting key 7ca4ac0be07b427a7879337e26f0e74b
I20260806 00:44:05.168185 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:05.156992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:59101 (local address 127.29.84.190:40753)
0806 00:44:05.157126 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:05.157129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:05.157829 (+ 700us) server_negotiation.cc:407] Connection header received
0806 00:44:05.159155 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:05.159158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:05.159208 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:05.159283 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:05.160299 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.160818 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.161922 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.162151 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.162247 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:05.164662 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:05.164665 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:05.164667 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:05.164692 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:05.166135 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.166644 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.166648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.166650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.166756 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.167122 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.167126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.167128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.167310 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:05.167427 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:05.167876 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 00:44:05.168022 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:44:05.168841 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:05.157251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40753 (local address 127.29.84.129:59101)
0806 00:44:05.157680 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:05.157719 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:05.158879 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:05.159431 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:05.159437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:05.159751 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:05.160154 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:05.160166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.160936 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.160938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:05.161778 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:05.161786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.162287 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.162289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:05.162417 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:05.162972 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:05.162987 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:05.164472 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:05.166258 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:05.166263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:05.166275 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:05.166537 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:05.166864 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:05.166867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:05.166869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:05.167004 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:05.167428 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:05.167436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:05.167644 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 00:44:05.167829 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 00:44:05.168126 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":135,"threads_started":1}
I20260806 00:44:05.169936 3086 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40753
I20260806 00:44:05.170157 3086 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:05.170595 3086 heartbeater.cc:511] Master 127.29.84.190:40753 requested a full tablet report, sending...
I20260806 00:44:05.172231 2802 ts_manager.cc:194] Registered new tserver with Master: e86ee1b10ec24869a066067cf8ea63c7 (127.29.84.129:35543)
I20260806 00:44:05.173854 2802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:59101
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:44:05.192242 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40753
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:05.299826 3095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:05.300047 3095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:05.300089 3095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:05.304174 3095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:05.304224 3095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:05.304311 3095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:05.309367 3095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:05.310282 3095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:05.311311 3095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:05.320634 3102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:05.320667 3105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:05.320639 3103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:05.321025 3095 server_base.cc:1034] running on GCE node
I20260806 00:44:05.321538 3095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:05.322091 3095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:05.323282 3095 hybrid_clock.cc:648] HybridClock initialized: now 1785977045323263 us; error 31 us; skew 500 ppm
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[2739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977045, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:05.326181 3095 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:44:05.327018 3095 webserver.cc:460] Webserver started at http://127.29.84.130:32853/ using document root <none> and password file <none>
I20260806 00:44:05.327606 3095 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:05.327700 3095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:05.327943 3095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:05.329849 3095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "7ac666843c35426680f34f46dca59f0b"
format_stamp: "Formatted at 2026-08-06 00:44:05 on dist-test-slave-n326"
server_key: "28e420302a5b3bc9f92728c545b33159"
server_key_iv: "c0104d7ed5daabf4931183820aabed1f"
server_key_version: "encryptionkey@0"
I20260806 00:44:05.330425 3095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "7ac666843c35426680f34f46dca59f0b"
format_stamp: "Formatted at 2026-08-06 00:44:05 on dist-test-slave-n326"
server_key: "28e420302a5b3bc9f92728c545b33159"
server_key_iv: "c0104d7ed5daabf4931183820aabed1f"
server_key_version: "encryptionkey@0"
I20260806 00:44:05.334031 3095 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:44:05.336336 3116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:05.337518 3095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:05.337654 3095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "7ac666843c35426680f34f46dca59f0b"
format_stamp: "Formatted at 2026-08-06 00:44:05 on dist-test-slave-n326"
server_key: "28e420302a5b3bc9f92728c545b33159"
server_key_iv: "c0104d7ed5daabf4931183820aabed1f"
server_key_version: "encryptionkey@0"
I20260806 00:44:05.337761 3095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:05.353788 3095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:05.356457 3095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:05.356648 3095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:05.357184 3095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:05.357950 3095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:05.358023 3095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:05.358090 3095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:05.358139 3095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:05.366588 3095 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:36185
I20260806 00:44:05.366654 3286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:36185 every 8 connection(s)
I20260806 00:44:05.367458 3095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:05.367965 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 3095
I20260806 00:44:05.368099 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:05.368396 30034 external_mini_cluster.cc:1428] Setting key 02ce0a1a007111e3d30d02ef6f991b73
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[2739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977045, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
I20260806 00:44:05.382197 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:05.369260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:38523 (local address 127.29.84.190:40753)
0806 00:44:05.369396 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:05.369400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:05.370232 (+ 832us) server_negotiation.cc:407] Connection header received
0806 00:44:05.371631 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:05.371637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:05.371700 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:05.371795 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:05.373313 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.373779 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.375563 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.375771 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.375851 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:05.378537 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:05.378543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:05.378548 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:05.378580 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:05.380001 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.380672 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.380675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.380676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.380783 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.381066 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.381068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.381069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.381255 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:05.381358 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:05.381838 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 00:44:05.382005 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:44:05.382987 3292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:05.369631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40753 (local address 127.29.84.130:38523)
0806 00:44:05.370074 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:05.370124 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:05.371436 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:05.372018 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:05.372030 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:05.372506 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:05.373166 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:05.373188 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.373915 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.373922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:05.375438 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:05.375452 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.375939 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.375946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:05.376182 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:05.376715 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:05.376751 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:05.378365 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:05.380140 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:05.380147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:05.380159 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:05.380591 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:05.380864 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:05.380865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:05.380867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:05.380988 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:05.381404 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:05.381409 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:05.381633 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 00:44:05.381780 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 00:44:05.381997 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":140,"threads_started":1}
I20260806 00:44:05.384238 3290 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40753
I20260806 00:44:05.384487 3290 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:05.385058 3290 heartbeater.cc:511] Master 127.29.84.190:40753 requested a full tablet report, sending...
I20260806 00:44:05.386126 2802 ts_manager.cc:194] Registered new tserver with Master: 7ac666843c35426680f34f46dca59f0b (127.29.84.130:36185)
I20260806 00:44:05.386689 2802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:38523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:44:05.398262 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40753
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:05.506337 3295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:05.506580 3295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:05.506676 3295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:05.510612 3295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:05.510689 3295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:05.510771 3295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:05.516355 3295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:05.517325 3295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:05.518275 3295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:05.527505 3304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:05.527626 3295 server_base.cc:1034] running on GCE node
W20260806 00:44:05.527519 3303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:05.527519 3307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:05.528189 3295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:05.528734 3295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:05.529925 3295 hybrid_clock.cc:648] HybridClock initialized: now 1785977045529912 us; error 33 us; skew 500 ppm
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[2739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977045, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:05.532617 3295 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:44:05.533394 3295 webserver.cc:460] Webserver started at http://127.29.84.131:36059/ using document root <none> and password file <none>
I20260806 00:44:05.533854 3295 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:05.533910 3295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:05.534057 3295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:05.535876 3295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "14816680b4ce46dba50175ffb7b7a87f"
format_stamp: "Formatted at 2026-08-06 00:44:05 on dist-test-slave-n326"
server_key: "59a905e5a32357e77b95399623c86143"
server_key_iv: "57066ae6c04068caef89065961f07521"
server_key_version: "encryptionkey@0"
I20260806 00:44:05.536342 3295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "14816680b4ce46dba50175ffb7b7a87f"
format_stamp: "Formatted at 2026-08-06 00:44:05 on dist-test-slave-n326"
server_key: "59a905e5a32357e77b95399623c86143"
server_key_iv: "57066ae6c04068caef89065961f07521"
server_key_version: "encryptionkey@0"
I20260806 00:44:05.539700 3295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 00:44:05.542068 3320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:05.543272 3295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:05.543392 3295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "14816680b4ce46dba50175ffb7b7a87f"
format_stamp: "Formatted at 2026-08-06 00:44:05 on dist-test-slave-n326"
server_key: "59a905e5a32357e77b95399623c86143"
server_key_iv: "57066ae6c04068caef89065961f07521"
server_key_version: "encryptionkey@0"
I20260806 00:44:05.543500 3295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:05.554531 3295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:05.557045 3295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:05.557178 3295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:05.557636 3295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:05.558344 3295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:05.558393 3295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:05.558435 3295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:05.558457 3295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:05.566886 3295 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:43799
I20260806 00:44:05.566941 3493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:43799 every 8 connection(s)
I20260806 00:44:05.567678 3295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:05.573853 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 3295
I20260806 00:44:05.573993 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:05.574278 30034 external_mini_cluster.cc:1428] Setting key 73832fcf89097dcd51bf13bc09e24b69
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[2739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977045, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:05.581029 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:05.569339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:57681 (local address 127.29.84.190:40753)
0806 00:44:05.569476 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:05.569480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:05.570232 (+ 752us) server_negotiation.cc:407] Connection header received
0806 00:44:05.571599 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:05.571603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:05.571661 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:05.571760 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:05.572818 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.573353 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.574673 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.575019 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.575179 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:05.577664 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:05.577667 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:05.577669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:05.577692 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:05.579008 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.579554 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.579557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.579558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.579668 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.579966 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.579981 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.579982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.580213 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:05.580290 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:05.580811 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 00:44:05.580917 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:44:05.581712 3498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:05.569671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40753 (local address 127.29.84.131:57681)
0806 00:44:05.570080 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:05.570130 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:05.571390 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:05.571915 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:05.571921 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:05.572247 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:05.572671 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:05.572683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.573487 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.573490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:05.574503 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:05.574513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.575228 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.575233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:05.575393 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:05.575997 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:05.576017 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:05.577520 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:05.579166 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:05.579170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:05.579180 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:05.579469 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:05.579753 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:05.579755 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:05.579756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:05.579882 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:05.580303 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:05.580307 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:05.580493 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 00:44:05.580723 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 00:44:05.580935 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":109,"threads_started":1}
I20260806 00:44:05.582787 3495 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40753
I20260806 00:44:05.583001 3495 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:05.583448 3495 heartbeater.cc:511] Master 127.29.84.190:40753 requested a full tablet report, sending...
I20260806 00:44:05.584424 2802 ts_manager.cc:194] Registered new tserver with Master: 14816680b4ce46dba50175ffb7b7a87f (127.29.84.131:43799)
I20260806 00:44:05.584970 2802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:57681
I20260806 00:44:05.587998 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[2739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[2739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:44:05.613508 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:05.604364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.29.84.129:35543)
0806 00:44:05.604719 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:05.604738 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:05.604758 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:44:05.604800 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:05.604803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:05.604948 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:05.605085 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:05.605779 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.606529 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.607433 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.607770 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.607894 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:05.610134 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:05.610143 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:05.610153 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:05.610175 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:05.611681 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.612117 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.612122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.612123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.612199 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.612504 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.612507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.612508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.612713 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:05.612943 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:05.613199 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 00:44:05.613349 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":108,"threads_started":1}
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[2739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:05.623533 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:05.615992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52190 (local address 127.29.84.190:40753)
0806 00:44:05.616172 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:05.616177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:05.616222 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:44:05.616347 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:05.616349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:05.616385 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:05.616448 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:05.616928 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.617495 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.618197 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:05.618456 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:05.618542 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:05.620703 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:05.620706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:05.620708 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:05.620728 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:05.622107 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.622453 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.622455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.622456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.622525 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:05.622750 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:05.622751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:05.622752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:05.622929 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:05.623021 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:05.623254 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 00:44:05.623335 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 00:44:05.623888 2802 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52190
I20260806 00:44:05.624478 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 2893
I20260806 00:44:05.638235 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 3095
I20260806 00:44:05.648545 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 3295
I20260806 00:44:05.656790 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 2757
2026-08-06T00:44:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2249 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[3514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[3514](info): set up 2 sockets
Aug 06 00:44:05 dist-test-slave-n326 krb5kdc[3514](info): commencing operation
krb5kdc: starting...
W20260806 00:44:06.871433 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:06 dist-test-slave-n326 krb5kdc[3514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:44:06.989851 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35921
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:35921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:07.094973 3531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:07.095258 3531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:07.095304 3531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:07.099223 3531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:07.099270 3531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:07.099289 3531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:07.099305 3531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:07.099320 3531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:07.104239 3531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:35921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:07.105021 3531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:07.105878 3531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:07.113711 3541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:07.113724 3543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:07.113785 3540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:07.114104 3531 server_base.cc:1034] running on GCE node
I20260806 00:44:07.114629 3531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:07.115672 3531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:07.116853 3531 hybrid_clock.cc:648] HybridClock initialized: now 1785977047116830 us; error 39 us; skew 500 ppm
Aug 06 00:44:07 dist-test-slave-n326 krb5kdc[3514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977047, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:07.119629 3531 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:44:07.120352 3531 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:07.120445 3531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:07.120675 3531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:07.122634 3531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "9f0f305e959143819ae5f49d920e1437"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "4666fce70fc2035e0b5669794f50421c"
server_key_iv: "3d20129f267ef7a2545b5aa4a7788e2e"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.123237 3531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "9f0f305e959143819ae5f49d920e1437"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "4666fce70fc2035e0b5669794f50421c"
server_key_iv: "3d20129f267ef7a2545b5aa4a7788e2e"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.126648 3531 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:44:07.129022 3549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.130165 3531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:07.130304 3531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "9f0f305e959143819ae5f49d920e1437"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "4666fce70fc2035e0b5669794f50421c"
server_key_iv: "3d20129f267ef7a2545b5aa4a7788e2e"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.130409 3531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:07.150738 3531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:07.153456 3531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:07.153651 3531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:07.160573 3531 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:35921
I20260806 00:44:07.160840 3622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:35921 every 8 connection(s)
I20260806 00:44:07.161267 3531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:07.163818 3624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:07.165112 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 3531
I20260806 00:44:07.165249 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:07.165539 30034 external_mini_cluster.cc:1428] Setting key 6c4cd6cd25e82974217c4353657a6836
I20260806 00:44:07.169535 3624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437: Bootstrap starting.
I20260806 00:44:07.171761 3624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:07.172519 3624 log.cc:824] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:44:07 dist-test-slave-n326 krb5kdc[3514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:07.174712 3624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437: No bootstrap required, opened a new log
I20260806 00:44:07.178920 3624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0f305e959143819ae5f49d920e1437" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35921 } }
I20260806 00:44:07.179183 3624 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:07.179258 3624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0f305e959143819ae5f49d920e1437, State: Initialized, Role: FOLLOWER
I20260806 00:44:07.179610 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.166941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.29.84.190:35921)
0806 00:44:07.167363 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:07.167378 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:07.167410 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:44:07.168008 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:07.168023 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:07.168325 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:07.168637 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:07.169439 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.170482 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.171404 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.171914 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.172101 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.175035 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:07.175040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:07.175051 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:07.175123 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:07.176720 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.177240 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.177244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.177248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.177352 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.177637 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.177639 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.177640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.177976 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:07.178138 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:07.178416 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 00:44:07.178716 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":138,"threads_started":1}
I20260806 00:44:07.179638 3624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0f305e959143819ae5f49d920e1437" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35921 } }
I20260806 00:44:07.179786 3624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:07.179845 3624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:07.179948 3624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:07.180908 3624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0f305e959143819ae5f49d920e1437" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35921 } }
I20260806 00:44:07.181223 3624 leader_election.cc:302] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f0f305e959143819ae5f49d920e1437; no voters:
I20260806 00:44:07.181583 3624 leader_election.cc:288] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:07.181653 3631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:07.182154 3631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [term 1 LEADER]: Becoming Leader. State: Replica: 9f0f305e959143819ae5f49d920e1437, State: Running, Role: LEADER
I20260806 00:44:07.182503 3631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0f305e959143819ae5f49d920e1437" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35921 } }
I20260806 00:44:07.182809 3624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:07.184257 3634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f0f305e959143819ae5f49d920e1437. Latest consensus state: current_term: 1 leader_uuid: "9f0f305e959143819ae5f49d920e1437" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0f305e959143819ae5f49d920e1437" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35921 } } }
I20260806 00:44:07.184297 3632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f0f305e959143819ae5f49d920e1437" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0f305e959143819ae5f49d920e1437" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35921 } } }
I20260806 00:44:07.184394 3634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:07.184418 3632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:07.184753 3657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:07.187326 3657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:07.192385 3657 catalog_manager.cc:1347] Generated new cluster ID: 6bdc918571874f58929a0a6089d0ba10
I20260806 00:44:07.192447 3657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:07.201160 3657 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:07.201996 3657 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:07.208729 3657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437: Generated new TSK 0
I20260806 00:44:07.209373 3657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:44:07.249864 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35921
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:07.368425 3673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:07.368661 3673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:07.368752 3673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:07.372594 3673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:07.372674 3673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:07.372758 3673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:07.377676 3673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:07.378557 3673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:07.379442 3673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:07.388284 3681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:07.388314 3683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:07.388314 3680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:07.388643 3673 server_base.cc:1034] running on GCE node
I20260806 00:44:07.389008 3673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:07.389561 3673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:07.390740 3673 hybrid_clock.cc:648] HybridClock initialized: now 1785977047390724 us; error 28 us; skew 500 ppm
Aug 06 00:44:07 dist-test-slave-n326 krb5kdc[3514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977047, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:07.393424 3673 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:44:07.394156 3673 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:07.394253 3673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:07.394474 3673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:07.396416 3673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "d58276a16c09475284f80658248558b4"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "96784f010769ce322f0d46384f06c036"
server_key_iv: "262fe520ff774bafa5eb5d38d91b1d25"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.396950 3673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "d58276a16c09475284f80658248558b4"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "96784f010769ce322f0d46384f06c036"
server_key_iv: "262fe520ff774bafa5eb5d38d91b1d25"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.400436 3673 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:07.402742 3692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.403956 3673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:07.404111 3673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "d58276a16c09475284f80658248558b4"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "96784f010769ce322f0d46384f06c036"
server_key_iv: "262fe520ff774bafa5eb5d38d91b1d25"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.404220 3673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:07.420724 3673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:07.423979 3673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:07.424194 3673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:07.424777 3673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:07.425576 3673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:07.425645 3673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.425715 3673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:07.425767 3673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.434973 3673 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:33749
I20260806 00:44:07.434995 3859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:33749 every 8 connection(s)
I20260806 00:44:07.435537 3673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:07.436088 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 3673
I20260806 00:44:07.436232 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:07.436573 30034 external_mini_cluster.cc:1428] Setting key bc52652b2d43e41805276c12652cea1c
Aug 06 00:44:07 dist-test-slave-n326 krb5kdc[3514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977047, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
I20260806 00:44:07.450784 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.437313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:54919 (local address 127.29.84.190:35921)
0806 00:44:07.437444 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:07.437449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:07.438533 (+ 1084us) server_negotiation.cc:407] Connection header received
0806 00:44:07.439968 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:07.439973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:07.440035 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:07.440098 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:07.441709 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.442225 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.444053 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.444278 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.444356 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.447145 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:07.447148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:07.447150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:07.447173 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:07.448705 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.449201 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.449205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.449206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.449285 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.449645 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.449649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.449650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.449847 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:07.449954 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:07.450473 (+ 519us) server_negotiation.cc:299] Negotiation successful
0806 00:44:07.450587 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:44:07.451537 3863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.437733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35921 (local address 127.29.84.129:54919)
0806 00:44:07.438399 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:07.438449 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:07.439713 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:07.440498 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:07.440506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:07.440993 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:07.441575 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.441588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.442358 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.442365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.443939 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.443950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.444437 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.444444 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.444691 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.445312 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:07.445337 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:07.446936 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:07.448833 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:07.448838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:07.448851 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:07.449093 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:07.449376 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:07.449380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:07.449382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:07.449513 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:07.449968 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:07.449976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:07.450235 (+ 259us) client_negotiation.cc:769] Sending connection context
0806 00:44:07.450432 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 00:44:07.450757 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":142,"threads_started":1}
I20260806 00:44:07.452772 3861 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35921
I20260806 00:44:07.452993 3861 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:07.453400 3861 heartbeater.cc:511] Master 127.29.84.190:35921 requested a full tablet report, sending...
I20260806 00:44:07.454768 3573 ts_manager.cc:194] Registered new tserver with Master: d58276a16c09475284f80658248558b4 (127.29.84.129:33749)
I20260806 00:44:07.455868 3573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:54919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:44:07.466172 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35921
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:07.573418 3869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:07.573655 3869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:07.573697 3869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:07.577610 3869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:07.577657 3869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:07.577713 3869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:07.582681 3869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:07.583606 3869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:07.584587 3869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:07.593349 3884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:07.593344 3881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:07.593345 3878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:07.593655 3869 server_base.cc:1034] running on GCE node
I20260806 00:44:07.594067 3869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:07.594686 3869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:07.595871 3869 hybrid_clock.cc:648] HybridClock initialized: now 1785977047595850 us; error 41 us; skew 500 ppm
Aug 06 00:44:07 dist-test-slave-n326 krb5kdc[3514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977047, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:07.598448 3869 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:44:07.599210 3869 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:07.599303 3869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:07.599527 3869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:07.601416 3869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "d5d324b555a34d0c8d1db75c656fb2c3"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "dba19ef07980d25b39e7dff186e4f097"
server_key_iv: "9425e455a3259c63238eb909b3db6c69"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.601944 3869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "d5d324b555a34d0c8d1db75c656fb2c3"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "dba19ef07980d25b39e7dff186e4f097"
server_key_iv: "9425e455a3259c63238eb909b3db6c69"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.605372 3869 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:44:07.607590 3890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.608784 3869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:07.608914 3869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "d5d324b555a34d0c8d1db75c656fb2c3"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "dba19ef07980d25b39e7dff186e4f097"
server_key_iv: "9425e455a3259c63238eb909b3db6c69"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.609017 3869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:07.633484 3869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:07.636247 3869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:07.636436 3869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:07.637017 3869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:07.637823 3869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:07.637904 3869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.637957 3869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:07.637980 3869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.646724 3869 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:34001
I20260806 00:44:07.646744 4052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:34001 every 8 connection(s)
I20260806 00:44:07.647337 3869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:07.651916 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 3869
I20260806 00:44:07.652038 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:07.652338 30034 external_mini_cluster.cc:1428] Setting key f18bb4da53aaf87113cdf5dbaccedabd
Aug 06 00:44:07 dist-test-slave-n326 krb5kdc[3514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977047, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:07.661020 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.648873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:46689 (local address 127.29.84.190:35921)
0806 00:44:07.649001 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:07.649005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:07.649714 (+ 709us) server_negotiation.cc:407] Connection header received
0806 00:44:07.650999 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:07.651003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:07.651081 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:07.651182 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:07.652361 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.653031 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.654269 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.654484 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.654596 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.657202 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:07.657208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:07.657214 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:07.657247 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:07.658906 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.659434 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.659439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.659441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.659536 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.659870 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.659875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.659876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.660130 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:07.660246 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:07.660684 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 00:44:07.660822 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:44:07.661603 4057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.649150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35921 (local address 127.29.84.130:46689)
0806 00:44:07.649577 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:07.649617 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:07.650799 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:07.651351 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:07.651357 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:07.651678 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:07.652221 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.652236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.653160 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.653164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.654136 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.654146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.654638 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.654642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.654800 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.655400 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:07.655416 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:07.657047 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:07.659020 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:07.659024 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:07.659034 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:07.659329 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:07.659647 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:07.659649 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:07.659651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:07.659773 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:07.660238 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:07.660243 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:07.660436 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 00:44:07.660586 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 00:44:07.660805 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
I20260806 00:44:07.662673 4053 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35921
I20260806 00:44:07.662909 4053 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:07.663414 4053 heartbeater.cc:511] Master 127.29.84.190:35921 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:44:07.664420 3573 ts_manager.cc:194] Registered new tserver with Master: d5d324b555a34d0c8d1db75c656fb2c3 (127.29.84.130:34001)
I20260806 00:44:07.665030 3573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:46689
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:44:07.682751 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35921
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:07.792694 4066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:07.792935 4066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:07.793007 4066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:07.796801 4066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:07.796872 4066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:07.796957 4066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:07.801980 4066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:07.802886 4066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:07.803872 4066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:07.813037 4075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:07.813056 4076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:07.813038 4081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:07.813334 4066 server_base.cc:1034] running on GCE node
I20260806 00:44:07.813732 4066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:07.814345 4066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:07.815533 4066 hybrid_clock.cc:648] HybridClock initialized: now 1785977047815515 us; error 29 us; skew 500 ppm
Aug 06 00:44:07 dist-test-slave-n326 krb5kdc[3514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977047, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:07.818234 4066 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:44:07.818899 4066 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:07.819005 4066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:07.819253 4066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:07.821144 4066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "495de5e3306b4d2980e015805b4984e4"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "d23eb139894ba41c9f4dc41cff76adaf"
server_key_iv: "2ad0011e0b4d7fb31175e3c845ba746a"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.821727 4066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "495de5e3306b4d2980e015805b4984e4"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "d23eb139894ba41c9f4dc41cff76adaf"
server_key_iv: "2ad0011e0b4d7fb31175e3c845ba746a"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.825286 4066 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:44:07.827527 4091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.828613 4066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:07.828754 4066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "495de5e3306b4d2980e015805b4984e4"
format_stamp: "Formatted at 2026-08-06 00:44:07 on dist-test-slave-n326"
server_key: "d23eb139894ba41c9f4dc41cff76adaf"
server_key_iv: "2ad0011e0b4d7fb31175e3c845ba746a"
server_key_version: "encryptionkey@0"
I20260806 00:44:07.828859 4066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:07.852026 4066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:07.854447 4066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:07.854640 4066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:07.855190 4066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:07.855938 4066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:07.856006 4066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.856074 4066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:07.856106 4066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:07.864682 4066 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:35727
I20260806 00:44:07.864703 4262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:35727 every 8 connection(s)
I20260806 00:44:07.865206 4066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:07.869306 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 4066
I20260806 00:44:07.869439 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:07.869774 30034 external_mini_cluster.cc:1428] Setting key f8149b13a3618e36b567ee36d55c8785
Aug 06 00:44:07 dist-test-slave-n326 krb5kdc[3514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977047, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:07.877715 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.866755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:54493 (local address 127.29.84.190:35921)
0806 00:44:07.866894 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:07.866899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:07.867777 (+ 878us) server_negotiation.cc:407] Connection header received
0806 00:44:07.868987 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:07.868990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:07.869038 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:07.869110 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:07.870464 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.871123 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.872435 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.872638 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.872715 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.874822 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:07.874825 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:07.874826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:07.874849 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:07.876083 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.876503 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.876505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.876507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.876574 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.876835 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.876837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.876838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.877041 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:07.877116 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:07.877513 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 00:44:07.877603 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:44:07.878430 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.867212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35921 (local address 127.29.84.131:54493)
0806 00:44:07.867636 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:07.867675 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:07.868789 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:07.869320 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:07.869330 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:07.869781 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:07.870311 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.870325 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.871254 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.871258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.872332 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.872342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.872758 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.872760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.872883 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.873350 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:07.873364 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:07.874685 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:07.876182 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:07.876186 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:07.876195 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:07.876423 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:07.876648 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:07.876650 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:07.876651 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:07.876761 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:07.877129 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:07.877133 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:07.877323 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 00:44:07.877462 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 00:44:07.877672 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":107,"threads_started":1}
I20260806 00:44:07.879415 4264 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35921
I20260806 00:44:07.879591 4264 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:07.879906 4264 heartbeater.cc:511] Master 127.29.84.190:35921 requested a full tablet report, sending...
I20260806 00:44:07.880781 3573 ts_manager.cc:194] Registered new tserver with Master: 495de5e3306b4d2980e015805b4984e4 (127.29.84.131:35727)
I20260806 00:44:07.881255 3573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:54493
I20260806 00:44:07.883503 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:44:07.892736 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.885407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37006 (local address 127.29.84.190:35921)
0806 00:44:07.885541 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:07.885546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:07.885754 (+ 208us) server_negotiation.cc:407] Connection header received
0806 00:44:07.885826 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:07.885828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:07.885863 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:07.885929 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:07.886691 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.887556 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.888366 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.888598 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.888688 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.889871 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:07.889874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:07.889876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:07.889899 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:07.891408 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.891783 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.891785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.891786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.891855 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:07.892097 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:07.892099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:07.892100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:07.892280 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:07.892408 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:07.892557 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 00:44:07.892638 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:44:07.895305 3573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:44:07.898140 3573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:07.912184 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.906589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:35727 (local address 127.0.0.1:53250)
0806 00:44:07.906981 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:07.907013 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:07.907127 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:07.907928 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:07.907931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:07.907951 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:07.908063 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.908069 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.909355 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.909359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.910647 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.910658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.911642 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.911646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.911764 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.911816 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 00:44:07.911920 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:44:07.912038 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":25,"thread_start_us":88,"threads_started":1}
I20260806 00:44:07.912333 4289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.906027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:33749 (local address 127.0.0.1:44312)
0806 00:44:07.906832 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:07.906845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:07.906924 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:07.908371 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:07.908374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:07.908391 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:07.908480 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.908486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.909698 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.909701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.910858 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.910865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.911886 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.911890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.911982 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.912005 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:44:07.912091 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:44:07.912157 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":23,"thread_start_us":87,"threads_started":1}
I20260806 00:44:07.912241 4291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.907115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53250 (local address 127.29.84.131:35727)
0806 00:44:07.907494 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:07.907505 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:07.907528 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:44:07.907601 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:07.907606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:07.907769 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:07.907948 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:07.908160 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.909215 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.910782 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.911512 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.911626 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.911758 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:07.911979 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 00:44:07.912089 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":156,"threads_started":1}
I20260806 00:44:07.912446 4290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.906674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.29.84.129:33749)
0806 00:44:07.907951 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:07.907959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:07.907983 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:44:07.908058 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:07.908063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:07.908249 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:07.908393 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:07.908594 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.909555 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.910967 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.911695 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.911834 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.912004 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:07.912124 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 00:44:07.912216 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":974,"thread_start_us":56,"threads_started":1}
I20260806 00:44:07.912842 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.905885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:34001 (local address 127.0.0.1:34580)
0806 00:44:07.906542 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:07.906592 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:07.906744 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:07.907614 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:07.907620 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:07.907642 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:07.907753 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.907759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.909265 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.909268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.910474 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:07.910485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.912465 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.912469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:07.912565 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.912588 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:44:07.912665 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 00:44:07.912728 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260806 00:44:07.913269 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:07.906044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.29.84.130:34001)
0806 00:44:07.906433 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:07.906449 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:07.906904 (+ 455us) server_negotiation.cc:407] Connection header received
0806 00:44:07.907011 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:07.907017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:07.907248 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:07.907418 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:07.907913 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.909124 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.910630 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:07.911512 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:07.911626 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:07.911759 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:07.913007 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0806 00:44:07.913106 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":113,"threads_started":1}
I20260806 00:44:07.915014 4167 tablet_service.cc:1467] Processing CreateTablet for tablet 2d03f80a7235403795ec1182b6e26fb4 (DEFAULT_TABLE table=test-table [id=fa0b3cbb70ed4d19b646cf6268790c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:07.915014 3763 tablet_service.cc:1467] Processing CreateTablet for tablet 2d03f80a7235403795ec1182b6e26fb4 (DEFAULT_TABLE table=test-table [id=fa0b3cbb70ed4d19b646cf6268790c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:07.915992 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d03f80a7235403795ec1182b6e26fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:07.916002 4167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d03f80a7235403795ec1182b6e26fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:07.921823 4293 tablet_bootstrap.cc:492] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: Bootstrap starting.
I20260806 00:44:07.921689 3959 tablet_service.cc:1467] Processing CreateTablet for tablet 2d03f80a7235403795ec1182b6e26fb4 (DEFAULT_TABLE table=test-table [id=fa0b3cbb70ed4d19b646cf6268790c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:07.922703 3959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d03f80a7235403795ec1182b6e26fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:07.923013 4294 tablet_bootstrap.cc:492] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4: Bootstrap starting.
I20260806 00:44:07.924304 4293 tablet_bootstrap.cc:654] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:07.925156 4294 tablet_bootstrap.cc:654] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:07.925464 4293 log.cc:824] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:07.926461 4294 log.cc:824] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:07.927349 4293 tablet_bootstrap.cc:492] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: No bootstrap required, opened a new log
I20260806 00:44:07.927556 4293 ts_tablet_manager.cc:1397] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 00:44:07.928486 4294 tablet_bootstrap.cc:492] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4: No bootstrap required, opened a new log
I20260806 00:44:07.928733 4294 ts_tablet_manager.cc:1397] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:44:07.929667 4299 tablet_bootstrap.cc:492] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: Bootstrap starting.
I20260806 00:44:07.931401 4293 raft_consensus.cc:348] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } }
I20260806 00:44:07.931713 4293 raft_consensus.cc:374] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:07.931813 4293 raft_consensus.cc:729] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495de5e3306b4d2980e015805b4984e4, State: Initialized, Role: FOLLOWER
I20260806 00:44:07.931838 4299 tablet_bootstrap.cc:654] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:07.932085 4294 raft_consensus.cc:348] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } }
I20260806 00:44:07.932308 4294 raft_consensus.cc:374] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:07.932377 4294 raft_consensus.cc:729] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58276a16c09475284f80658248558b4, State: Initialized, Role: FOLLOWER
I20260806 00:44:07.932374 4293 consensus_queue.cc:260] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } }
I20260806 00:44:07.932909 4294 consensus_queue.cc:260] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } }
I20260806 00:44:07.933137 4299 log.cc:824] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:07.933460 4293 ts_tablet_manager.cc:1428] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:44:07.933851 4264 heartbeater.cc:503] Master 127.29.84.190:35921 was elected leader, sending a full tablet report...
I20260806 00:44:07.933852 4294 ts_tablet_manager.cc:1428] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:44:07.934244 3861 heartbeater.cc:503] Master 127.29.84.190:35921 was elected leader, sending a full tablet report...
I20260806 00:44:07.935128 4299 tablet_bootstrap.cc:492] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: No bootstrap required, opened a new log
I20260806 00:44:07.935359 4299 ts_tablet_manager.cc:1397] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
W20260806 00:44:07.936517 3862 tablet.cc:2369] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:44:07.938933 4299 raft_consensus.cc:348] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } }
I20260806 00:44:07.939188 4299 raft_consensus.cc:374] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:07.939266 4299 raft_consensus.cc:729] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d324b555a34d0c8d1db75c656fb2c3, State: Initialized, Role: FOLLOWER
I20260806 00:44:07.939620 4299 consensus_queue.cc:260] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } }
I20260806 00:44:07.940201 4053 heartbeater.cc:503] Master 127.29.84.190:35921 was elected leader, sending a full tablet report...
I20260806 00:44:07.940465 4299 ts_tablet_manager.cc:1428] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:44:08.098387 4301 raft_consensus.cc:482] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:08.098585 4301 raft_consensus.cc:504] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } }
I20260806 00:44:08.099771 4301 leader_election.cc:288] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5d324b555a34d0c8d1db75c656fb2c3 (127.29.84.130:34001), 495de5e3306b4d2980e015805b4984e4 (127.29.84.131:35727)
I20260806 00:44:08.103515 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.100078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:54545 (local address 127.29.84.130:34001)
0806 00:44:08.100303 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:08.100310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:08.100333 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:44:08.100506 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:08.100509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:08.100553 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:08.100632 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:08.100895 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.101408 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.102535 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.102955 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.103112 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:08.103168 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:08.103340 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 00:44:08.103407 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260806 00:44:08.103546 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.099961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:34001 (local address 127.29.84.129:54545)
0806 00:44:08.100267 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:08.100282 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:08.100390 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:08.100654 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:08.100657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:08.100674 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:08.100792 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:08.100797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.101549 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.101552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:08.102413 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:08.102422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.103103 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.103107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:08.103200 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:08.103228 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:44:08.103312 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 00:44:08.103383 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":41,"thread_start_us":102,"threads_started":1}
I20260806 00:44:08.103694 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.100022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:35727 (local address 127.29.84.129:41525)
0806 00:44:08.100443 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:08.100455 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:08.100574 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:08.100810 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:08.100813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:08.100828 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:08.100946 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:08.100957 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.101699 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.101705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:08.102557 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:08.102566 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.103256 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.103259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:08.103374 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:08.103395 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:44:08.103487 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:44:08.103554 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":39,"thread_start_us":157,"threads_started":1}
I20260806 00:44:08.103696 4291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.100169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:41525 (local address 127.29.84.131:35727)
0806 00:44:08.100296 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:08.100302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:08.100545 (+ 243us) server_negotiation.cc:407] Connection header received
0806 00:44:08.100674 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:08.100677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:08.100717 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:08.100799 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:08.101069 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.101576 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.102687 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.103142 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.103260 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:08.103313 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:08.103514 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 00:44:08.103581 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:44:08.104240 3994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d03f80a7235403795ec1182b6e26fb4" candidate_uuid: "d58276a16c09475284f80658248558b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" is_pre_election: true
I20260806 00:44:08.104285 4191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d03f80a7235403795ec1182b6e26fb4" candidate_uuid: "d58276a16c09475284f80658248558b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495de5e3306b4d2980e015805b4984e4" is_pre_election: true
I20260806 00:44:08.104537 3994 raft_consensus.cc:2454] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58276a16c09475284f80658248558b4 in term 0.
I20260806 00:44:08.104590 4191 raft_consensus.cc:2454] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58276a16c09475284f80658248558b4 in term 0.
I20260806 00:44:08.105011 3695 leader_election.cc:302] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d58276a16c09475284f80658248558b4, d5d324b555a34d0c8d1db75c656fb2c3; no voters:
I20260806 00:44:08.105255 4301 raft_consensus.cc:2781] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:08.105340 4301 raft_consensus.cc:482] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:08.105382 4301 raft_consensus.cc:3037] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:08.106376 4301 raft_consensus.cc:504] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } }
I20260806 00:44:08.106704 4301 leader_election.cc:288] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [CANDIDATE]: Term 1 election: Requested vote from peers d5d324b555a34d0c8d1db75c656fb2c3 (127.29.84.130:34001), 495de5e3306b4d2980e015805b4984e4 (127.29.84.131:35727)
I20260806 00:44:08.107059 3994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d03f80a7235403795ec1182b6e26fb4" candidate_uuid: "d58276a16c09475284f80658248558b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d324b555a34d0c8d1db75c656fb2c3"
I20260806 00:44:08.107143 4191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d03f80a7235403795ec1182b6e26fb4" candidate_uuid: "d58276a16c09475284f80658248558b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495de5e3306b4d2980e015805b4984e4"
I20260806 00:44:08.107240 4191 raft_consensus.cc:3037] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:08.107249 3994 raft_consensus.cc:3037] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:08.108529 4191 raft_consensus.cc:2454] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58276a16c09475284f80658248558b4 in term 1.
I20260806 00:44:08.108529 3994 raft_consensus.cc:2454] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58276a16c09475284f80658248558b4 in term 1.
I20260806 00:44:08.108834 3694 leader_election.cc:302] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 495de5e3306b4d2980e015805b4984e4, d58276a16c09475284f80658248558b4; no voters:
I20260806 00:44:08.108992 4301 raft_consensus.cc:2781] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:08.109225 4301 raft_consensus.cc:686] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 1 LEADER]: Becoming Leader. State: Replica: d58276a16c09475284f80658248558b4, State: Running, Role: LEADER
I20260806 00:44:08.109553 4301 consensus_queue.cc:237] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } }
I20260806 00:44:08.112449 3570 catalog_manager.cc:5495] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d58276a16c09475284f80658248558b4 (127.29.84.129). New cstate: current_term: 1 leader_uuid: "d58276a16c09475284f80658248558b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58276a16c09475284f80658248558b4" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 } health_report { overall_health: UNKNOWN } } }
W20260806 00:44:08.116003 4265 tablet.cc:2369] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:44:08.144515 4290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.141306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44316 (local address 127.29.84.129:33749)
0806 00:44:08.141427 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:08.141434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:08.141451 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:44:08.141588 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:08.141590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:08.141636 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:08.141720 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:08.141964 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.142465 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.143315 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:08.143522 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:08.143615 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:08.143863 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:08.144225 (+ 362us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:08.144311 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:08.144363 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 00:44:08.144424 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 00:44:08.148311 4054 tablet.cc:2369] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:44:08.149992 3994 raft_consensus.cc:1261] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [term 1 FOLLOWER]: Refusing update from remote peer d58276a16c09475284f80658248558b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:08.149995 4191 raft_consensus.cc:1261] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [term 1 FOLLOWER]: Refusing update from remote peer d58276a16c09475284f80658248558b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:08.150671 4309 consensus_queue.cc:1035] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "495de5e3306b4d2980e015805b4984e4" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:08.150839 4301 consensus_queue.cc:1035] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d324b555a34d0c8d1db75c656fb2c3" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:08.159298 4317 mvcc.cc:205] Tried to move back new op lower bound from 7315361989215293440 to 7315361989059035136. Current Snapshot: MvccSnapshot[applied={T|T < 7315361989215293440}]
I20260806 00:44:08.169052 3570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:08.169222 3570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:08.171875 3570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f0f305e959143819ae5f49d920e1437: Sending DeleteTablet for 3 replicas of tablet 2d03f80a7235403795ec1182b6e26fb4
I20260806 00:44:08.172611 3763 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d03f80a7235403795ec1182b6e26fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:08 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:44312
I20260806 00:44:08.172616 3959 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d03f80a7235403795ec1182b6e26fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:08 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:34580
I20260806 00:44:08.173122 4331 tablet_replica.cc:330] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4: stopping tablet replica
I20260806 00:44:08.173307 4332 tablet_replica.cc:330] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: stopping tablet replica
I20260806 00:44:08.173409 4167 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d03f80a7235403795ec1182b6e26fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:08 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53250
I20260806 00:44:08.173647 4332 raft_consensus.cc:2229] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:08.173902 4332 raft_consensus.cc:2258] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:08.174333 4331 raft_consensus.cc:2229] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:44:08.174679 4331 raft_consensus.cc:2258] T 2d03f80a7235403795ec1182b6e26fb4 P d58276a16c09475284f80658248558b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:08.174877 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 3673
I20260806 00:44:08.179402 4333 tablet_replica.cc:330] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: stopping tablet replica
I20260806 00:44:08.179817 4333 raft_consensus.cc:2229] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:08.180191 4333 raft_consensus.cc:2258] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:08.181279 4332 ts_tablet_manager.cc:1905] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:08.183370 4333 ts_tablet_manager.cc:1905] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:08.184222 4332 ts_tablet_manager.cc:1918] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:08.184330 4332 log.cc:1196] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/2d03f80a7235403795ec1182b6e26fb4
I20260806 00:44:08.184656 4332 ts_tablet_manager.cc:1939] T 2d03f80a7235403795ec1182b6e26fb4 P d5d324b555a34d0c8d1db75c656fb2c3: Deleting consensus metadata
I20260806 00:44:08.185729 3551 catalog_manager.cc:4841] TS d5d324b555a34d0c8d1db75c656fb2c3 (127.29.84.130:34001): tablet 2d03f80a7235403795ec1182b6e26fb4 (table test-table [id=fa0b3cbb70ed4d19b646cf6268790c12]) successfully deleted
I20260806 00:44:08.186580 4333 ts_tablet_manager.cc:1918] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:08.186681 4333 log.cc:1196] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/2d03f80a7235403795ec1182b6e26fb4
I20260806 00:44:08.187012 4333 ts_tablet_manager.cc:1939] T 2d03f80a7235403795ec1182b6e26fb4 P 495de5e3306b4d2980e015805b4984e4: Deleting consensus metadata
I20260806 00:44:08.188210 3550 catalog_manager.cc:4841] TS 495de5e3306b4d2980e015805b4984e4 (127.29.84.131:35727): tablet 2d03f80a7235403795ec1182b6e26fb4 (table test-table [id=fa0b3cbb70ed4d19b646cf6268790c12]) successfully deleted
W20260806 00:44:08.188974 3550 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:33749 (error 108)
I20260806 00:44:08.189028 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 3869
I20260806 00:44:08.198660 4289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.198410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:33749 (local address 127.0.0.1:44328)
0806 00:44:08.198535 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:08.198594 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 00:44:08.198813 3550 catalog_manager.cc:4584] TS d58276a16c09475284f80658248558b4 (127.29.84.129:33749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d03f80a7235403795ec1182b6e26fb4: Network error: Client connection negotiation failed: client connection to 127.29.84.129:33749: connect: Connection refused (error 111)
I20260806 00:44:08.204952 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 4066
I20260806 00:44:08.215588 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 3531
2026-08-06T00:44:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2563 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T00:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:08Z Disabled control of system clock
I20260806 00:44:08.251760 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:36799
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:36943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:36799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 00:44:08.355849 4340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:08.356063 4340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:08.356096 4340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:08.359941 4340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:08.359985 4340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:08.360001 4340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:08.360018 4340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:08.360033 4340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:08.364291 4340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:36943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:36799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:36799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:08.365049 4340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:08.366117 4340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:08.374210 4351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:08.374187 4349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:08.374231 4340 server_base.cc:1034] running on GCE node
W20260806 00:44:08.374184 4354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:08.374939 4340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:08.375962 4340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:08.377135 4340 hybrid_clock.cc:648] HybridClock initialized: now 1785977048377109 us; error 36 us; skew 500 ppm
I20260806 00:44:08.378775 4340 webserver.cc:460] Webserver started at http://127.29.84.190:35497/ using document root <none> and password file <none>
I20260806 00:44:08.379372 4340 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:08.379436 4340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:08.379645 4340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:08.381497 4340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "66710f6fb9b344529e45705f2a203c29"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "1a62ee152abd571a22d1d5c451329542"
server_key_iv: "065562d859e5cab925ca47a4a49a8976"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.382037 4340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "66710f6fb9b344529e45705f2a203c29"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "1a62ee152abd571a22d1d5c451329542"
server_key_iv: "065562d859e5cab925ca47a4a49a8976"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.385413 4340 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:44:08.387578 4362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.388656 4340 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:08.388777 4340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "66710f6fb9b344529e45705f2a203c29"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "1a62ee152abd571a22d1d5c451329542"
server_key_iv: "065562d859e5cab925ca47a4a49a8976"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.388877 4340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:08.393123 4340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:08.393661 4340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:08.393838 4340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:08.400345 4340 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:36799
I20260806 00:44:08.400352 4437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:36799 every 8 connection(s)
I20260806 00:44:08.401222 4340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:08.404402 4438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:08.407275 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 4340
I20260806 00:44:08.407397 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:08.407693 30034 external_mini_cluster.cc:1428] Setting key 3048c43f00977d3008fbffee7b18bf68
I20260806 00:44:08.410344 4438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29: Bootstrap starting.
I20260806 00:44:08.412756 4438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:08.413136 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.409098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.29.84.190:36799)
0806 00:44:08.409715 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:08.409730 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:08.409763 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:44:08.410372 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:08.410388 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:08.410763 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:08.411123 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:08.411899 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:08.411903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:08.411914 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:08.411925 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:08.411980 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:08.411995 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:08.412153 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:08.412226 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 00:44:08.412456 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":134,"threads_started":1}
I20260806 00:44:08.413546 4438 log.cc:824] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:08.415562 4438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29: No bootstrap required, opened a new log
I20260806 00:44:08.419150 4438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66710f6fb9b344529e45705f2a203c29" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36799 } }
I20260806 00:44:08.419399 4438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:08.419483 4438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66710f6fb9b344529e45705f2a203c29, State: Initialized, Role: FOLLOWER
I20260806 00:44:08.419906 4438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66710f6fb9b344529e45705f2a203c29" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36799 } }
I20260806 00:44:08.420044 4438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:08.420132 4438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:08.420257 4438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:08.421135 4438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66710f6fb9b344529e45705f2a203c29" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36799 } }
I20260806 00:44:08.421483 4438 leader_election.cc:302] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66710f6fb9b344529e45705f2a203c29; no voters:
I20260806 00:44:08.421824 4438 leader_election.cc:288] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:08.421932 4450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:08.422185 4450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [term 1 LEADER]: Becoming Leader. State: Replica: 66710f6fb9b344529e45705f2a203c29, State: Running, Role: LEADER
I20260806 00:44:08.422519 4450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66710f6fb9b344529e45705f2a203c29" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36799 } }
I20260806 00:44:08.423007 4438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:08.424203 4454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66710f6fb9b344529e45705f2a203c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66710f6fb9b344529e45705f2a203c29" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36799 } } }
I20260806 00:44:08.424345 4454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:08.424580 4455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66710f6fb9b344529e45705f2a203c29. Latest consensus state: current_term: 1 leader_uuid: "66710f6fb9b344529e45705f2a203c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66710f6fb9b344529e45705f2a203c29" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36799 } } }
I20260806 00:44:08.424732 4455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:08.424755 4468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:08.427979 4468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:08.433333 4468 catalog_manager.cc:1347] Generated new cluster ID: 88517538781c4a67bf0a6cb009f3f0b5
I20260806 00:44:08.433400 4468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:08.460268 4468 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:08.460544 4468 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:08.465983 4468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29: Generated new TSK 0
I20260806 00:44:08.466658 4468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:44:08.478464 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36799
--builtin_ntp_servers=127.29.84.148:36943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 00:44:08.586290 4484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:08.586505 4484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:08.586541 4484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:08.590505 4484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:08.590551 4484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:08.590629 4484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:08.595333 4484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:36943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:08.596269 4484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:08.597493 4484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:08.606961 4497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:08.607048 4491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:08.607133 4492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:08.607255 4484 server_base.cc:1034] running on GCE node
I20260806 00:44:08.607697 4484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:08.608278 4484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:08.609483 4484 hybrid_clock.cc:648] HybridClock initialized: now 1785977048609471 us; error 24 us; skew 500 ppm
I20260806 00:44:08.611359 4484 webserver.cc:460] Webserver started at http://127.29.84.129:38085/ using document root <none> and password file <none>
I20260806 00:44:08.611950 4484 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:08.612043 4484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:08.612263 4484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:08.614142 4484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "ac83cdd4b694362f5618fb79546dc2f8"
server_key_iv: "798b3b63c78965fc2cea2d0104a8b956"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.614707 4484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "ac83cdd4b694362f5618fb79546dc2f8"
server_key_iv: "798b3b63c78965fc2cea2d0104a8b956"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.618108 4484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:44:08.620461 4508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.621534 4484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:08.621676 4484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "ac83cdd4b694362f5618fb79546dc2f8"
server_key_iv: "798b3b63c78965fc2cea2d0104a8b956"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.621785 4484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:08.626410 4484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:08.626951 4484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:08.627149 4484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:08.627661 4484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:08.628502 4484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:08.628574 4484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.628644 4484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:08.628690 4484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.636813 4484 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:39039
I20260806 00:44:08.636857 4675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:39039 every 8 connection(s)
I20260806 00:44:08.637614 4484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:08.643039 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.639314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:52989 (local address 127.29.84.190:36799)
0806 00:44:08.639465 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:08.639470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:08.640451 (+ 981us) server_negotiation.cc:407] Connection header received
0806 00:44:08.641698 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:08.641700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:08.641735 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:08.641806 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:08.642712 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:08.642715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:08.642717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:08.642727 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:08.642746 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:08.642753 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:08.642827 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:08.642905 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 00:44:08.642938 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 00:44:08.643808 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.639706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36799 (local address 127.29.84.129:52989)
0806 00:44:08.640320 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:08.640355 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:08.641523 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:08.641918 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:08.641923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:08.641999 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:08.642484 (+ 485us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:08.642485 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:08.642520 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:08.642579 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:08.642811 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:08.642815 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:08.642818 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:44:08.642930 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 00:44:08.643165 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":119,"threads_started":1}
I20260806 00:44:08.644867 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 4484
I20260806 00:44:08.644925 4677 heartbeater.cc:347] Connected to a master server at 127.29.84.190:36799
I20260806 00:44:08.645000 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:08.645133 4677 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:08.645340 30034 external_mini_cluster.cc:1428] Setting key 86a9e7fe9cbe1c057c32d1537e47e8d2
I20260806 00:44:08.645565 4677 heartbeater.cc:511] Master 127.29.84.190:36799 requested a full tablet report, sending...
I20260806 00:44:08.647024 4380 ts_manager.cc:194] Registered new tserver with Master: d7b4f55120ff4c93b94d74ec96bcdcc6 (127.29.84.129:39039)
I20260806 00:44:08.647291 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36799
--builtin_ntp_servers=127.29.84.148:36943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 00:44:08.754489 4682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:08.754699 4682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:08.754735 4682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:08.758580 4682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:08.758625 4682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:08.758670 4682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:08.763444 4682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:36943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:08.764335 4682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:08.765460 4682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:08.774320 4692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:08.774290 4690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:08.774382 4682 server_base.cc:1034] running on GCE node
W20260806 00:44:08.774298 4688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:08.774837 4682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:08.775481 4682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:08.776670 4682 hybrid_clock.cc:648] HybridClock initialized: now 1785977048776662 us; error 30 us; skew 500 ppm
I20260806 00:44:08.778331 4682 webserver.cc:460] Webserver started at http://127.29.84.130:44305/ using document root <none> and password file <none>
I20260806 00:44:08.778860 4682 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:08.778955 4682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:08.779230 4682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:08.781018 4682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "9cb08096c0694fc9b6642359f949d58e"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "abfa02281e907a954969a3bd75aaea32"
server_key_iv: "0760cd6bd55adbe29f10b1c7f105f388"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.781596 4682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "9cb08096c0694fc9b6642359f949d58e"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "abfa02281e907a954969a3bd75aaea32"
server_key_iv: "0760cd6bd55adbe29f10b1c7f105f388"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.784943 4682 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:44:08.787031 4700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.788198 4682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:08.788341 4682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "9cb08096c0694fc9b6642359f949d58e"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "abfa02281e907a954969a3bd75aaea32"
server_key_iv: "0760cd6bd55adbe29f10b1c7f105f388"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.788444 4682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:08.793391 4682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:08.793927 4682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:08.794106 4682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:08.794584 4682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:08.795456 4682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:08.795526 4682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.795596 4682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:08.795639 4682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.804253 4682 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:43011
I20260806 00:44:08.804328 4879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:43011 every 8 connection(s)
I20260806 00:44:08.805048 4682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:08.810570 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.806717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:45337 (local address 127.29.84.190:36799)
0806 00:44:08.806852 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:08.806856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:08.807783 (+ 927us) server_negotiation.cc:407] Connection header received
0806 00:44:08.809015 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:08.809019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:08.809065 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:08.809144 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:08.810096 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:08.810099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:08.810102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:08.810113 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:08.810136 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:08.810149 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:08.810238 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:08.810388 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 00:44:08.810431 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:44:08.811424 4885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.806966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36799 (local address 127.29.84.130:45337)
0806 00:44:08.807646 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:08.807682 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:08.808812 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:08.809288 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:08.809294 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:08.809372 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:08.809846 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:08.809848 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:08.809890 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:08.809949 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:08.810232 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:08.810242 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:08.810249 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 00:44:08.810409 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 00:44:08.810631 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":186,"threads_started":1}
I20260806 00:44:08.812536 4881 heartbeater.cc:347] Connected to a master server at 127.29.84.190:36799
I20260806 00:44:08.812752 4881 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:08.813122 4881 heartbeater.cc:511] Master 127.29.84.190:36799 requested a full tablet report, sending...
I20260806 00:44:08.813534 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 4682
I20260806 00:44:08.813651 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:08.813926 30034 external_mini_cluster.cc:1428] Setting key 81d0280234ba50bf634389975f80c018
I20260806 00:44:08.814217 4380 ts_manager.cc:194] Registered new tserver with Master: 9cb08096c0694fc9b6642359f949d58e (127.29.84.130:43011)
I20260806 00:44:08.815891 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36799
--builtin_ntp_servers=127.29.84.148:36943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 00:44:08.923148 4887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:08.923403 4887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:08.923491 4887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:08.927991 4887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:08.928040 4887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:08.928092 4887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:08.932801 4887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:36943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:08.933665 4887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:08.934868 4887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:08.944069 4893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:08.944103 4899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:08.944059 4894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:08.944221 4887 server_base.cc:1034] running on GCE node
I20260806 00:44:08.944584 4887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:08.945147 4887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:08.946326 4887 hybrid_clock.cc:648] HybridClock initialized: now 1785977048946305 us; error 46 us; skew 500 ppm
I20260806 00:44:08.948017 4887 webserver.cc:460] Webserver started at http://127.29.84.131:41421/ using document root <none> and password file <none>
I20260806 00:44:08.948571 4887 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:08.948661 4887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:08.948882 4887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:08.950711 4887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "c22484f41f1e41639bf6c70f0fc5ad69"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "62dc4aa46aba2de8e8e89729ca7adaba"
server_key_iv: "d68406b609ec41f4edb6733aae52b7e8"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.951301 4887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "c22484f41f1e41639bf6c70f0fc5ad69"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "62dc4aa46aba2de8e8e89729ca7adaba"
server_key_iv: "d68406b609ec41f4edb6733aae52b7e8"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.954816 4887 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:44:08.957250 4908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.958362 4887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:08.958500 4887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "c22484f41f1e41639bf6c70f0fc5ad69"
format_stamp: "Formatted at 2026-08-06 00:44:08 on dist-test-slave-n326"
server_key: "62dc4aa46aba2de8e8e89729ca7adaba"
server_key_iv: "d68406b609ec41f4edb6733aae52b7e8"
server_key_version: "encryptionkey@0"
I20260806 00:44:08.958611 4887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:08.962982 4887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:08.963541 4887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:08.963717 4887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:08.964229 4887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:08.965057 4887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:08.965128 4887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.965190 4887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:08.965242 4887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:08.973536 4887 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:35443
I20260806 00:44:08.973584 5074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:35443 every 8 connection(s)
I20260806 00:44:08.974392 4887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:08.979913 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.976063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:55299 (local address 127.29.84.190:36799)
0806 00:44:08.976196 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:08.976200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:08.977180 (+ 980us) server_negotiation.cc:407] Connection header received
0806 00:44:08.978381 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:08.978383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:08.978428 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:08.978504 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:08.979535 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:08.979538 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:08.979540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:08.979551 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:08.979571 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:08.979579 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:08.979658 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:08.979773 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 00:44:08.979815 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:44:08.980655 5080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.976438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36799 (local address 127.29.84.131:55299)
0806 00:44:08.977027 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:08.977066 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:08.978170 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:08.978648 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:08.978654 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:08.978732 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:08.979305 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:08.979306 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:08.979341 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:08.979402 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:08.979656 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:08.979663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:08.979667 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:44:08.979785 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 00:44:08.979949 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":105,"threads_started":1}
I20260806 00:44:08.981773 5075 heartbeater.cc:347] Connected to a master server at 127.29.84.190:36799
I20260806 00:44:08.981983 5075 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:08.982187 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 4887
I20260806 00:44:08.982322 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:08.982407 5075 heartbeater.cc:511] Master 127.29.84.190:36799 requested a full tablet report, sending...
I20260806 00:44:08.982664 30034 external_mini_cluster.cc:1428] Setting key 48f6608e409007c2c2c2bd03e050f090
I20260806 00:44:08.983569 4381 ts_manager.cc:194] Registered new tserver with Master: c22484f41f1e41639bf6c70f0fc5ad69 (127.29.84.131:35443)
I20260806 00:44:08.984845 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:44:08.988294 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:08.986670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.29.84.190:36799)
0806 00:44:08.986822 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:08.986826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:08.986984 (+ 158us) server_negotiation.cc:407] Connection header received
0806 00:44:08.987178 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:08.987182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:08.987221 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:08.987307 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:08.987909 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:08.987912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:08.987915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:08.987927 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:08.987946 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:08.987954 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:08.988054 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:08.988126 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 00:44:08.988166 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 00:44:08.990796 4381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:44:08.994041 4381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:09.004733 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.001639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:35443 (local address 127.0.0.1:50024)
0806 00:44:09.002363 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:09.002414 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:09.002525 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:09.003179 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:09.003184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:09.003457 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:09.003967 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:09.003971 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:09.003993 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:09.004061 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:09.004379 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:09.004388 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:09.004393 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:44:09.004495 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:44:09.004591 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"spinlock_wait_cycles":3968,"thread_start_us":78,"threads_started":1}
I20260806 00:44:09.004757 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.001761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.29.84.131:35443)
0806 00:44:09.002139 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:09.002151 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:09.002572 (+ 421us) server_negotiation.cc:407] Connection header received
0806 00:44:09.002652 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:09.002657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:09.002823 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:09.002954 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:09.004204 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:09.004209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:09.004220 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:09.004231 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:09.004274 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:09.004294 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:09.004403 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:09.004506 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 00:44:09.004612 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":106,"threads_started":1}
I20260806 00:44:09.004973 5098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.002283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59508 (local address 127.29.84.130:43011)
0806 00:44:09.003270 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:09.003287 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:09.003311 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:44:09.003364 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:09.003369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:09.003534 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:09.003689 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:09.004291 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:09.004295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:09.004303 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:09.004311 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:09.004339 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:09.004354 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:09.004476 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:09.004595 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 00:44:09.004686 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":102,"threads_started":1}
I20260806 00:44:09.005823 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.002116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:43011 (local address 127.0.0.1:59508)
0806 00:44:09.002361 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:09.002423 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:09.002526 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:09.003663 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:09.003666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:09.003680 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:09.004095 (+ 415us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:09.004097 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:09.004108 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:09.004136 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:09.004450 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:09.004453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:09.004456 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:44:09.005235 (+ 779us) client_negotiation.cc:240] Negotiation successful
0806 00:44:09.005271 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"spinlock_wait_cycles":2304,"thread_start_us":94,"threads_started":1}
I20260806 00:44:09.006718 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.002297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:39039 (local address 127.0.0.1:58502)
0806 00:44:09.002917 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:09.002929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:09.002998 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:09.005197 (+ 2199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:09.005200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:09.005232 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:09.005657 (+ 425us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:09.005659 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:09.005669 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:09.005693 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:09.006473 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:09.006477 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:09.006481 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:44:09.006571 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 00:44:09.006605 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":92,"threads_started":1}
I20260806 00:44:09.007716 4975 tablet_service.cc:1467] Processing CreateTablet for tablet 2329509b451249d2a712d7188729be30 (DEFAULT_TABLE table=test-table [id=044214553e0a483aaba146094d8f37a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:09.008630 4975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2329509b451249d2a712d7188729be30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:09.008344 4771 tablet_service.cc:1467] Processing CreateTablet for tablet 2329509b451249d2a712d7188729be30 (DEFAULT_TABLE table=test-table [id=044214553e0a483aaba146094d8f37a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:09.009336 4771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2329509b451249d2a712d7188729be30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:09.013049 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.002733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58502 (local address 127.29.84.129:39039)
0806 00:44:09.004729 (+ 1996us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:09.004741 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:09.004764 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:44:09.004814 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:09.004821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:09.005017 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:09.005447 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:09.006292 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:09.006298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:09.006307 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:09.006316 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:09.006349 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:09.006371 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:09.006474 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:09.006599 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 00:44:09.006664 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1880,"thread_start_us":119,"threads_started":1}
I20260806 00:44:09.014817 5104 tablet_bootstrap.cc:492] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: Bootstrap starting.
I20260806 00:44:09.015832 5105 tablet_bootstrap.cc:492] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: Bootstrap starting.
I20260806 00:44:09.015712 4577 tablet_service.cc:1467] Processing CreateTablet for tablet 2329509b451249d2a712d7188729be30 (DEFAULT_TABLE table=test-table [id=044214553e0a483aaba146094d8f37a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:09.016551 4577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2329509b451249d2a712d7188729be30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:09.017074 5104 tablet_bootstrap.cc:654] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:09.017863 5105 tablet_bootstrap.cc:654] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:09.017949 5104 log.cc:824] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:09.018626 5105 log.cc:824] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:09.019940 5104 tablet_bootstrap.cc:492] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: No bootstrap required, opened a new log
I20260806 00:44:09.020128 5104 ts_tablet_manager.cc:1397] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:44:09.020562 5105 tablet_bootstrap.cc:492] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: No bootstrap required, opened a new log
I20260806 00:44:09.020736 5105 ts_tablet_manager.cc:1397] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 00:44:09.022464 5110 tablet_bootstrap.cc:492] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6: Bootstrap starting.
I20260806 00:44:09.024631 5110 tablet_bootstrap.cc:654] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:09.024967 5104 raft_consensus.cc:348] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } }
I20260806 00:44:09.025157 5105 raft_consensus.cc:348] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } }
I20260806 00:44:09.025297 5104 raft_consensus.cc:374] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:09.025353 5105 raft_consensus.cc:374] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:09.025357 5104 raft_consensus.cc:729] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22484f41f1e41639bf6c70f0fc5ad69, State: Initialized, Role: FOLLOWER
I20260806 00:44:09.025405 5105 raft_consensus.cc:729] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb08096c0694fc9b6642359f949d58e, State: Initialized, Role: FOLLOWER
I20260806 00:44:09.025401 5110 log.cc:824] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:09.025790 5104 consensus_queue.cc:260] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } }
I20260806 00:44:09.025790 5105 consensus_queue.cc:260] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } }
I20260806 00:44:09.026722 5104 ts_tablet_manager.cc:1428] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:44:09.026722 5105 ts_tablet_manager.cc:1428] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:44:09.027055 4881 heartbeater.cc:503] Master 127.29.84.190:36799 was elected leader, sending a full tablet report...
I20260806 00:44:09.027053 5075 heartbeater.cc:503] Master 127.29.84.190:36799 was elected leader, sending a full tablet report...
I20260806 00:44:09.027264 5110 tablet_bootstrap.cc:492] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6: No bootstrap required, opened a new log
I20260806 00:44:09.027415 5110 ts_tablet_manager.cc:1397] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:44:09.031721 5110 raft_consensus.cc:348] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } }
I20260806 00:44:09.031925 5110 raft_consensus.cc:374] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:09.031981 5110 raft_consensus.cc:729] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b4f55120ff4c93b94d74ec96bcdcc6, State: Initialized, Role: FOLLOWER
I20260806 00:44:09.032364 5110 consensus_queue.cc:260] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } }
I20260806 00:44:09.032941 4677 heartbeater.cc:503] Master 127.29.84.190:36799 was elected leader, sending a full tablet report...
I20260806 00:44:09.033212 5110 ts_tablet_manager.cc:1428] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260806 00:44:09.055936 4882 tablet.cc:2369] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:44:09.138685 4679 tablet.cc:2369] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:44:09.225366 5079 tablet.cc:2369] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:44:09.261184 5114 raft_consensus.cc:482] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:09.261473 5114 raft_consensus.cc:504] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } }
I20260806 00:44:09.262661 5114 leader_election.cc:288] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c22484f41f1e41639bf6c70f0fc5ad69 (127.29.84.131:35443), 9cb08096c0694fc9b6642359f949d58e (127.29.84.130:43011)
I20260806 00:44:09.264972 5098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.263095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:40439 (local address 127.29.84.130:43011)
0806 00:44:09.263251 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:09.263255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:09.263354 (+ 99us) server_negotiation.cc:407] Connection header received
0806 00:44:09.263502 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:09.263505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:09.263552 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:09.263649 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:09.264548 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:09.264552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:09.264554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:09.264567 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:09.264590 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:09.264601 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:09.264694 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:09.264797 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 00:44:09.264839 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:44:09.264972 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.263314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:44333 (local address 127.29.84.131:35443)
0806 00:44:09.263481 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:09.263485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:09.263501 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:44:09.263540 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:09.263543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:09.263574 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:09.263663 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:09.264548 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:09.264552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:09.264554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:09.264567 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:09.264589 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:09.264606 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:09.264694 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:09.264797 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 00:44:09.264838 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:44:09.265030 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.262946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:43011 (local address 127.29.84.129:40439)
0806 00:44:09.263251 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:09.263268 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:09.263380 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:09.263700 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:09.263703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:09.263720 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:09.264352 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:09.264353 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:09.264361 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:09.264376 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:09.264695 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:09.264698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:09.264702 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:44:09.264783 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 00:44:09.264850 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":18,"thread_start_us":114,"threads_started":1}
I20260806 00:44:09.265196 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.262946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:35443 (local address 127.29.84.129:44333)
0806 00:44:09.263380 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:09.263393 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:09.263463 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:09.263700 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:09.263703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:09.263720 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:09.264327 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:09.264331 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:09.264344 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:09.264376 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:09.264695 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:09.264698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:09.264701 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:44:09.264783 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 00:44:09.264836 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":14,"thread_start_us":106,"threads_started":1}
I20260806 00:44:09.265805 4796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2329509b451249d2a712d7188729be30" candidate_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb08096c0694fc9b6642359f949d58e" is_pre_election: true
I20260806 00:44:09.265890 5005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2329509b451249d2a712d7188729be30" candidate_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" is_pre_election: true
I20260806 00:44:09.266114 4796 raft_consensus.cc:2454] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7b4f55120ff4c93b94d74ec96bcdcc6 in term 0.
I20260806 00:44:09.266214 5005 raft_consensus.cc:2454] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7b4f55120ff4c93b94d74ec96bcdcc6 in term 0.
I20260806 00:44:09.266649 4513 leader_election.cc:302] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9cb08096c0694fc9b6642359f949d58e, c22484f41f1e41639bf6c70f0fc5ad69, d7b4f55120ff4c93b94d74ec96bcdcc6; no voters:
I20260806 00:44:09.266945 5114 raft_consensus.cc:2781] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:09.267046 5114 raft_consensus.cc:482] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:09.267136 5114 raft_consensus.cc:3037] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:09.268199 5114 raft_consensus.cc:504] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } }
I20260806 00:44:09.268539 5114 leader_election.cc:288] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [CANDIDATE]: Term 1 election: Requested vote from peers c22484f41f1e41639bf6c70f0fc5ad69 (127.29.84.131:35443), 9cb08096c0694fc9b6642359f949d58e (127.29.84.130:43011)
I20260806 00:44:09.268898 4796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2329509b451249d2a712d7188729be30" candidate_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb08096c0694fc9b6642359f949d58e"
I20260806 00:44:09.268899 5005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2329509b451249d2a712d7188729be30" candidate_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22484f41f1e41639bf6c70f0fc5ad69"
I20260806 00:44:09.269027 4796 raft_consensus.cc:3037] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:09.269027 5005 raft_consensus.cc:3037] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:09.270293 5005 raft_consensus.cc:2454] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b4f55120ff4c93b94d74ec96bcdcc6 in term 1.
I20260806 00:44:09.270293 4796 raft_consensus.cc:2454] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b4f55120ff4c93b94d74ec96bcdcc6 in term 1.
I20260806 00:44:09.270566 4513 leader_election.cc:302] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9cb08096c0694fc9b6642359f949d58e, c22484f41f1e41639bf6c70f0fc5ad69, d7b4f55120ff4c93b94d74ec96bcdcc6; no voters:
I20260806 00:44:09.270749 5114 raft_consensus.cc:2781] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:09.271019 5114 raft_consensus.cc:686] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [term 1 LEADER]: Becoming Leader. State: Replica: d7b4f55120ff4c93b94d74ec96bcdcc6, State: Running, Role: LEADER
I20260806 00:44:09.271368 5114 consensus_queue.cc:237] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } }
I20260806 00:44:09.274287 4380 catalog_manager.cc:5495] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7b4f55120ff4c93b94d74ec96bcdcc6 (127.29.84.129). New cstate: current_term: 1 leader_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7b4f55120ff4c93b94d74ec96bcdcc6" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 39039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260806 00:44:09.293643 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.292123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58504 (local address 127.29.84.129:39039)
0806 00:44:09.292261 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:09.292266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:09.292280 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:44:09.292373 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:09.292376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:09.292518 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:09.292620 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:09.293236 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:09.293239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:09.293242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:09.293253 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:09.293274 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:09.293285 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:09.293381 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:09.293469 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 00:44:09.293506 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:44:09.299258 5005 raft_consensus.cc:1261] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [term 1 FOLLOWER]: Refusing update from remote peer d7b4f55120ff4c93b94d74ec96bcdcc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:09.299258 4796 raft_consensus.cc:1261] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [term 1 FOLLOWER]: Refusing update from remote peer d7b4f55120ff4c93b94d74ec96bcdcc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:09.299996 5114 consensus_queue.cc:1035] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb08096c0694fc9b6642359f949d58e" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:09.300205 5119 consensus_queue.cc:1035] T 2329509b451249d2a712d7188729be30 P d7b4f55120ff4c93b94d74ec96bcdcc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c22484f41f1e41639bf6c70f0fc5ad69" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:09.306843 5124 mvcc.cc:205] Tried to move back new op lower bound from 7315361993922351104 to 7315361993817690112. Current Snapshot: MvccSnapshot[applied={T|T < 7315361993922351104}]
I20260806 00:44:09.309036 5125 mvcc.cc:205] Tried to move back new op lower bound from 7315361993922351104 to 7315361993817690112. Current Snapshot: MvccSnapshot[applied={T|T < 7315361993922351104}]
I20260806 00:44:09.314575 4380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:09.314742 4380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:09.317133 4380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66710f6fb9b344529e45705f2a203c29: Sending DeleteTablet for 3 replicas of tablet 2329509b451249d2a712d7188729be30
I20260806 00:44:09.317912 4577 tablet_service.cc:1514] Processing DeleteTablet for tablet 2329509b451249d2a712d7188729be30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:09 UTC) from {username='slave'} at 127.0.0.1:58502
I20260806 00:44:09.317921 4975 tablet_service.cc:1514] Processing DeleteTablet for tablet 2329509b451249d2a712d7188729be30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:09 UTC) from {username='slave'} at 127.0.0.1:50024
I20260806 00:44:09.318354 4771 tablet_service.cc:1514] Processing DeleteTablet for tablet 2329509b451249d2a712d7188729be30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:09 UTC) from {username='slave'} at 127.0.0.1:59508
I20260806 00:44:09.318614 5138 tablet_replica.cc:330] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: stopping tablet replica
I20260806 00:44:09.318719 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 4484
I20260806 00:44:09.318746 5139 tablet_replica.cc:330] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: stopping tablet replica
I20260806 00:44:09.321466 5139 raft_consensus.cc:2229] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:09.321725 5139 raft_consensus.cc:2258] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:09.326673 5139 ts_tablet_manager.cc:1905] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:09.327163 5138 raft_consensus.cc:2229] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:09.327425 5138 raft_consensus.cc:2258] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:09.329278 5138 ts_tablet_manager.cc:1905] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:09.329394 5139 ts_tablet_manager.cc:1918] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:09.329487 5139 log.cc:1196] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/2329509b451249d2a712d7188729be30
I20260806 00:44:09.329798 5139 ts_tablet_manager.cc:1939] T 2329509b451249d2a712d7188729be30 P 9cb08096c0694fc9b6642359f949d58e: Deleting consensus metadata
I20260806 00:44:09.330947 4366 catalog_manager.cc:4841] TS 9cb08096c0694fc9b6642359f949d58e (127.29.84.130:43011): tablet 2329509b451249d2a712d7188729be30 (table test-table [id=044214553e0a483aaba146094d8f37a8]) successfully deleted
I20260806 00:44:09.331754 5138 ts_tablet_manager.cc:1918] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:09.331820 5138 log.cc:1196] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/2329509b451249d2a712d7188729be30
I20260806 00:44:09.332036 5138 ts_tablet_manager.cc:1939] T 2329509b451249d2a712d7188729be30 P c22484f41f1e41639bf6c70f0fc5ad69: Deleting consensus metadata
I20260806 00:44:09.332881 4363 catalog_manager.cc:4841] TS c22484f41f1e41639bf6c70f0fc5ad69 (127.29.84.131:35443): tablet 2329509b451249d2a712d7188729be30 (table test-table [id=044214553e0a483aaba146094d8f37a8]) successfully deleted
W20260806 00:44:09.358024 4363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:39039 (error 108)
I20260806 00:44:09.358130 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 4682
I20260806 00:44:09.359998 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.359477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:39039 (local address 127.0.0.1:58508)
0806 00:44:09.359725 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:09.359830 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260806 00:44:09.360239 4363 catalog_manager.cc:4584] TS d7b4f55120ff4c93b94d74ec96bcdcc6 (127.29.84.129:39039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2329509b451249d2a712d7188729be30: Network error: Client connection negotiation failed: client connection to 127.29.84.129:39039: connect: Connection refused (error 111)
I20260806 00:44:09.371126 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 4887
I20260806 00:44:09.378641 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 4340
2026-08-06T00:44:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1161 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 00:44:09.405285 30034 mini_oidc.cc:152] Starting JWKS server
I20260806 00:44:09.406899 30034 webserver.cc:460] Webserver started at https://127.0.0.1:34657/ using document root <none> and password file <none>
I20260806 00:44:09.407035 30034 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 00:44:09.407397 30034 webserver.cc:460] Webserver started at http://127.0.0.1:33257/ using document root <none> and password file <none>
2026-08-06T00:44:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:09Z Disabled control of system clock
I20260806 00:44:09.417209 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38133
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:38929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:38133
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34657/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/testchainca.cert with env {}
W20260806 00:44:09.524190 5152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:09.524425 5152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:09.524463 5152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:09.528249 5152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:09.528291 5152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:09.528307 5152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:09.528323 5152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:09.528352 5152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:44:09.528370 5152 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:44:09.528390 5152 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34657/jwks.json
I20260806 00:44:09.533243 5152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:38133
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34657/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:09.534051 5152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:09.534906 5152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:09.543021 5159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:09.543053 5164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:09.543056 5162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:09.543308 5152 server_base.cc:1034] running on GCE node
I20260806 00:44:09.543898 5152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:09.544910 5152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:09.546065 5152 hybrid_clock.cc:648] HybridClock initialized: now 1785977049546049 us; error 28 us; skew 500 ppm
I20260806 00:44:09.547816 5152 webserver.cc:460] Webserver started at http://127.29.84.190:44937/ using document root <none> and password file <none>
I20260806 00:44:09.548364 5152 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:09.548425 5152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:09.548657 5152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:09.550490 5152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "1cec65c8a8204a3d91e4391c45dd74a0"
format_stamp: "Formatted at 2026-08-06 00:44:09 on dist-test-slave-n326"
server_key: "d5c01774a590a7ad178a7b48e223bccf"
server_key_iv: "c12b60f89ada1fb8b249d53de678a91c"
server_key_version: "encryptionkey@0"
I20260806 00:44:09.551039 5152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "1cec65c8a8204a3d91e4391c45dd74a0"
format_stamp: "Formatted at 2026-08-06 00:44:09 on dist-test-slave-n326"
server_key: "d5c01774a590a7ad178a7b48e223bccf"
server_key_iv: "c12b60f89ada1fb8b249d53de678a91c"
server_key_version: "encryptionkey@0"
I20260806 00:44:09.554602 5152 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:44:09.556845 5181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:09.558027 5152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:44:09.558149 5152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "1cec65c8a8204a3d91e4391c45dd74a0"
format_stamp: "Formatted at 2026-08-06 00:44:09 on dist-test-slave-n326"
server_key: "d5c01774a590a7ad178a7b48e223bccf"
server_key_iv: "c12b60f89ada1fb8b249d53de678a91c"
server_key_version: "encryptionkey@0"
I20260806 00:44:09.558249 5152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:09.586922 5152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:09.587556 5152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:09.587764 5152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:09.594214 5152 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:38133
I20260806 00:44:09.594223 5264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:38133 every 8 connection(s)
I20260806 00:44:09.595109 5152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:09.598234 5265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:09.602849 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 5152
I20260806 00:44:09.603008 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:09.603341 30034 external_mini_cluster.cc:1428] Setting key ffea3d5e8fba8d873da05162c80996e5
I20260806 00:44:09.604470 5265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0: Bootstrap starting.
I20260806 00:44:09.606806 5265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:09.607730 5265 log.cc:824] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:09.609625 5265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0: No bootstrap required, opened a new log
I20260806 00:44:09.612613 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.604762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.29.84.190:38133)
0806 00:44:09.605400 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:09.605413 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:09.605445 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:44:09.606073 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:09.606098 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:09.606460 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:09.606776 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:09.607265 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:09.608461 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:09.609351 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:09.609882 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:09.610047 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:09.610916 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:09.610921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:09.610932 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:09.611380 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:09.611432 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:09.611452 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:09.611562 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:09.611715 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 00:44:09.611953 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":127,"threads_started":1}
I20260806 00:44:09.613894 5265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cec65c8a8204a3d91e4391c45dd74a0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38133 } }
I20260806 00:44:09.614126 5265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:09.614210 5265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cec65c8a8204a3d91e4391c45dd74a0, State: Initialized, Role: FOLLOWER
I20260806 00:44:09.614712 5265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cec65c8a8204a3d91e4391c45dd74a0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38133 } }
I20260806 00:44:09.614886 5265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:09.614959 5265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:09.615077 5265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:09.615597 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:44:09.616133 5265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cec65c8a8204a3d91e4391c45dd74a0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38133 } }
I20260806 00:44:09.616489 5265 leader_election.cc:302] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cec65c8a8204a3d91e4391c45dd74a0; no voters:
I20260806 00:44:09.616803 5265 leader_election.cc:288] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:09.616933 5275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:09.617177 5275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [term 1 LEADER]: Becoming Leader. State: Replica: 1cec65c8a8204a3d91e4391c45dd74a0, State: Running, Role: LEADER
I20260806 00:44:09.617483 5275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cec65c8a8204a3d91e4391c45dd74a0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38133 } }
I20260806 00:44:09.618000 5265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:09.619026 5280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cec65c8a8204a3d91e4391c45dd74a0. Latest consensus state: current_term: 1 leader_uuid: "1cec65c8a8204a3d91e4391c45dd74a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cec65c8a8204a3d91e4391c45dd74a0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38133 } } }
I20260806 00:44:09.619175 5280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:09.619494 5279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cec65c8a8204a3d91e4391c45dd74a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cec65c8a8204a3d91e4391c45dd74a0" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38133 } } }
I20260806 00:44:09.619585 5279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:09.619679 5291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:09.622598 5291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:09.628224 5291 catalog_manager.cc:1347] Generated new cluster ID: 608936a929d34f728b12ba425e7705ba
I20260806 00:44:09.628291 5291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:09.660405 5291 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:09.661653 5291 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:09.666867 5291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cec65c8a8204a3d91e4391c45dd74a0: Generated new TSK 0
I20260806 00:44:09.667625 5291 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 00:44:09.702158 5310 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 00:44:09.702315 5310 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 00:44:09.702864 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.685855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.29.84.190:38133)
0806 00:44:09.685986 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:09.685991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:09.686183 (+ 192us) server_negotiation.cc:407] Connection header received
0806 00:44:09.686365 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:09.686368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:09.686423 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:09.686535 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:44:09.686752 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:09.687589 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:09.688441 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:09.688818 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:09.688921 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:09.689259 (+ 338us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:44:09.702481 (+ 13222us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34657/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 00:44:09.702708 (+ 227us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51468: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34657/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 00:44:09.702941 5270 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51468: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34657/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 00:44:09.703938 5309 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:09.685727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38133 (local address 127.0.0.1:51468)
0806 00:44:09.686098 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:09.686110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:09.686210 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:09.686548 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:09.686550 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:09.686556 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:44:09.686655 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:09.686660 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:09.687715 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:09.687718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:09.688333 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:09.688340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:09.688930 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:09.688933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:09.689001 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:09.689056 (+ 55us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:44:09.702666 (+ 13610us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34657/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 00:44:09.702829 (+ 163us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.84.190:38133: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34657/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":134,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.84.190:38133: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34657/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 00:44:09.704612 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 5152
2026-08-06T00:44:09Z chronyd exiting
I20260806 00:44:09.750945 30034 test_util.cc:175] -----------------------------------------------
I20260806 00:44:09.751019 30034 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977020829091-30034-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (347 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 00:44:09.752131 30034 mini_oidc.cc:152] Starting JWKS server
I20260806 00:44:09.752907 30034 webserver.cc:460] Webserver started at https://127.0.0.1:34229/ using document root <none> and password file <none>
I20260806 00:44:09.752982 30034 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 00:44:09.753230 30034 webserver.cc:460] Webserver started at http://127.0.0.1:41341/ using document root <none> and password file <none>
2026-08-06T00:44:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:09Z Disabled control of system clock
I20260806 00:44:09.762745 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:44931
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:42935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:44931
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34229/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/testchainca.cert with env {}
W20260806 00:44:09.868571 5319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:09.868778 5319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:09.868813 5319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:09.872442 5319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:09.872479 5319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:09.872493 5319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:09.872507 5319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:09.872521 5319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:44:09.872534 5319 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:44:09.872546 5319 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34229/jwks.json
I20260806 00:44:09.877210 5319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:42935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:44931
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:44931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34229/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:09.878000 5319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:09.878855 5319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:44:09.886782 5319 server_base.cc:1034] running on GCE node
W20260806 00:44:09.886722 5325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:09.886765 5327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:09.886734 5333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:09.887593 5319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:09.888597 5319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:09.889794 5319 hybrid_clock.cc:648] HybridClock initialized: now 1785977049889775 us; error 31 us; skew 500 ppm
I20260806 00:44:09.891583 5319 webserver.cc:460] Webserver started at http://127.29.84.190:39347/ using document root <none> and password file <none>
I20260806 00:44:09.892113 5319 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:09.892171 5319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:09.892429 5319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:09.894196 5319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "0869f675a0c0487089cc73f0f4a3b4fa"
format_stamp: "Formatted at 2026-08-06 00:44:09 on dist-test-slave-n326"
server_key: "86ba4addf32c9317134620f51bb2832c"
server_key_iv: "cbea1173434eb2e640cf5e28a96f80d5"
server_key_version: "encryptionkey@0"
I20260806 00:44:09.894726 5319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "0869f675a0c0487089cc73f0f4a3b4fa"
format_stamp: "Formatted at 2026-08-06 00:44:09 on dist-test-slave-n326"
server_key: "86ba4addf32c9317134620f51bb2832c"
server_key_iv: "cbea1173434eb2e640cf5e28a96f80d5"
server_key_version: "encryptionkey@0"
I20260806 00:44:09.898104 5319 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:44:09.900359 5340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:09.901525 5319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:09.901652 5319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "0869f675a0c0487089cc73f0f4a3b4fa"
format_stamp: "Formatted at 2026-08-06 00:44:09 on dist-test-slave-n326"
server_key: "86ba4addf32c9317134620f51bb2832c"
server_key_iv: "cbea1173434eb2e640cf5e28a96f80d5"
server_key_version: "encryptionkey@0"
I20260806 00:44:09.901748 5319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:09.919826 5319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:09.920324 5319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:09.920492 5319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:09.926604 5319 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:44931
I20260806 00:44:09.926621 5420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:44931 every 8 connection(s)
I20260806 00:44:09.927485 5319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:09.927790 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 5319
I20260806 00:44:09.927915 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:09.928257 30034 external_mini_cluster.cc:1428] Setting key ac9060f7d906b93d396c0adf3198a906
I20260806 00:44:09.930682 5421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:09.935774 5421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa: Bootstrap starting.
I20260806 00:44:09.936635 5426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:09.929516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.29.84.190:44931)
0806 00:44:09.930099 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:09.930119 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:09.930150 (+ 31us) server_negotiation.cc:407] Connection header received
0806 00:44:09.930744 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:09.930767 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:09.931194 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:09.931498 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:09.931896 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:09.932785 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:09.933691 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:09.934113 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:09.934272 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:09.935142 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:09.935148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:09.935163 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:09.935503 (+ 340us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:09.935553 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:09.935571 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:09.935685 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:09.935811 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 00:44:09.936037 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":142,"threads_started":1}
I20260806 00:44:09.937829 5421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:09.938665 5421 log.cc:824] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:09.939179 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:44:09.940331 5421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa: No bootstrap required, opened a new log
I20260806 00:44:09.943576 5421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869f675a0c0487089cc73f0f4a3b4fa" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44931 } }
I20260806 00:44:09.943776 5421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:09.943850 5421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0869f675a0c0487089cc73f0f4a3b4fa, State: Initialized, Role: FOLLOWER
I20260806 00:44:09.944240 5421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869f675a0c0487089cc73f0f4a3b4fa" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44931 } }
I20260806 00:44:09.944379 5421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:09.944468 5421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:09.944581 5421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:09.945336 5421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869f675a0c0487089cc73f0f4a3b4fa" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44931 } }
I20260806 00:44:09.945636 5421 leader_election.cc:302] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0869f675a0c0487089cc73f0f4a3b4fa; no voters:
I20260806 00:44:09.945919 5421 leader_election.cc:288] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:09.945981 5430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:09.946165 5430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [term 1 LEADER]: Becoming Leader. State: Replica: 0869f675a0c0487089cc73f0f4a3b4fa, State: Running, Role: LEADER
I20260806 00:44:09.946441 5430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869f675a0c0487089cc73f0f4a3b4fa" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44931 } }
I20260806 00:44:09.947024 5421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:09.948112 5434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0869f675a0c0487089cc73f0f4a3b4fa. Latest consensus state: current_term: 1 leader_uuid: "0869f675a0c0487089cc73f0f4a3b4fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869f675a0c0487089cc73f0f4a3b4fa" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44931 } } }
I20260806 00:44:09.948138 5432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0869f675a0c0487089cc73f0f4a3b4fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869f675a0c0487089cc73f0f4a3b4fa" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44931 } } }
I20260806 00:44:09.948230 5434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:09.948248 5432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:09.949153 5445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:09.951521 5445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:09.956393 5445 catalog_manager.cc:1347] Generated new cluster ID: 88d649e4c58c43f1bb68ec12482f1b03
I20260806 00:44:09.956459 5445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:09.976992 5445 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:09.977805 5445 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:09.986256 5445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0869f675a0c0487089cc73f0f4a3b4fa: Generated new TSK 0
I20260806 00:44:09.986812 5445 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 00:44:10.020334 5473 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 00:44:10.020406 5473 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 00:44:10.020933 5426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:10.000319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57774 (local address 127.29.84.190:44931)
0806 00:44:10.000446 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:10.000451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:10.000645 (+ 194us) server_negotiation.cc:407] Connection header received
0806 00:44:10.000783 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:10.000786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:10.000826 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:10.000904 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:44:10.001146 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.001642 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.002810 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.002988 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.003052 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:10.003352 (+ 300us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:44:10.020667 (+ 17315us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 00:44:10.020843 (+ 176us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57774: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 00:44:10.021008 5426 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57774: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 00:44:10.021163 5471 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:10.000274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44931 (local address 127.0.0.1:57774)
0806 00:44:10.000557 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:10.000571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:10.000675 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:10.000933 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:10.000936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:10.000943 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:44:10.001048 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:10.001054 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.001741 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.001745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:10.002700 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:10.002710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.003109 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.003113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:10.003189 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:10.003214 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:44:10.020829 (+ 17615us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 00:44:10.021000 (+ 171us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.84.190:44931: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":110,"threads_started":1}
I20260806 00:44:10.021731 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 5319
2026-08-06T00:44:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (316 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 00:44:10.068851 30034 mini_oidc.cc:152] Starting JWKS server
I20260806 00:44:10.069667 30034 webserver.cc:460] Webserver started at https://127.0.0.1:44823/ using document root <none> and password file <none>
I20260806 00:44:10.069802 30034 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 00:44:10.070078 30034 webserver.cc:460] Webserver started at http://127.0.0.1:46811/ using document root <none> and password file <none>
2026-08-06T00:44:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:10Z Disabled control of system clock
I20260806 00:44:10.079417 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:46517
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:37023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:46517
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44823/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 00:44:10.186267 5483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:10.186488 5483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:10.186524 5483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:10.190387 5483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:10.190435 5483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:10.190450 5483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:10.190465 5483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:10.190521 5483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:44:10.190543 5483 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:44:10.190577 5483 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44823/jwks.json
I20260806 00:44:10.194784 5483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:37023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:46517
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:46517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44823/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:10.195683 5483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:10.196928 5483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:10.205834 5494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:10.205827 5493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:10.205966 5496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:10.206630 5483 server_base.cc:1034] running on GCE node
I20260806 00:44:10.207319 5483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:10.208581 5483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:10.209769 5483 hybrid_clock.cc:648] HybridClock initialized: now 1785977050209748 us; error 36 us; skew 500 ppm
I20260806 00:44:10.211607 5483 webserver.cc:460] Webserver started at http://127.29.84.190:38879/ using document root <none> and password file <none>
I20260806 00:44:10.212141 5483 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:10.212232 5483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:10.212457 5483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:10.214308 5483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "1e37cb5439c040998bbf5a5022d31b5b"
format_stamp: "Formatted at 2026-08-06 00:44:10 on dist-test-slave-n326"
server_key: "4a18cbfbf51bf238d648561873e8ac13"
server_key_iv: "d58066ab5617b45a3ad29961bce1d113"
server_key_version: "encryptionkey@0"
I20260806 00:44:10.214890 5483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "1e37cb5439c040998bbf5a5022d31b5b"
format_stamp: "Formatted at 2026-08-06 00:44:10 on dist-test-slave-n326"
server_key: "4a18cbfbf51bf238d648561873e8ac13"
server_key_iv: "d58066ab5617b45a3ad29961bce1d113"
server_key_version: "encryptionkey@0"
I20260806 00:44:10.218308 5483 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:44:10.220669 5505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:10.221858 5483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:10.222009 5483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "1e37cb5439c040998bbf5a5022d31b5b"
format_stamp: "Formatted at 2026-08-06 00:44:10 on dist-test-slave-n326"
server_key: "4a18cbfbf51bf238d648561873e8ac13"
server_key_iv: "d58066ab5617b45a3ad29961bce1d113"
server_key_version: "encryptionkey@0"
I20260806 00:44:10.222121 5483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:10.241555 5483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:10.242122 5483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:10.242300 5483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:10.248855 5483 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:46517
I20260806 00:44:10.248893 5585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:46517 every 8 connection(s)
I20260806 00:44:10.249668 5483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:10.252632 5586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:10.254472 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 5483
I20260806 00:44:10.254606 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:10.254877 30034 external_mini_cluster.cc:1428] Setting key 6032e1d1df31d812fc627c3259c28639
I20260806 00:44:10.258865 5586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b: Bootstrap starting.
I20260806 00:44:10.261303 5586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:10.262176 5586 log.cc:824] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:10.264225 5586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b: No bootstrap required, opened a new log
I20260806 00:44:10.264839 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:10.256162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41556 (local address 127.29.84.190:46517)
0806 00:44:10.256870 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:10.256890 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:10.256930 (+ 40us) server_negotiation.cc:407] Connection header received
0806 00:44:10.257769 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:10.257791 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:10.258261 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:10.258627 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:10.259030 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.260488 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.261336 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.262043 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.262219 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:10.262990 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:10.262995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:10.263008 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:10.263476 (+ 468us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:10.263543 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:10.263569 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:10.263664 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:10.263768 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 00:44:10.264002 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":115,"threads_started":1}
I20260806 00:44:10.267441 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:44:10.267838 5586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e37cb5439c040998bbf5a5022d31b5b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46517 } }
I20260806 00:44:10.268030 5586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:10.268097 5586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e37cb5439c040998bbf5a5022d31b5b, State: Initialized, Role: FOLLOWER
I20260806 00:44:10.268585 5586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e37cb5439c040998bbf5a5022d31b5b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46517 } }
I20260806 00:44:10.268739 5586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:10.268815 5586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:10.268903 5586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:10.269829 5586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e37cb5439c040998bbf5a5022d31b5b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46517 } }
I20260806 00:44:10.270156 5586 leader_election.cc:302] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e37cb5439c040998bbf5a5022d31b5b; no voters:
I20260806 00:44:10.270476 5586 leader_election.cc:288] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:10.270613 5594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:10.270846 5594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [term 1 LEADER]: Becoming Leader. State: Replica: 1e37cb5439c040998bbf5a5022d31b5b, State: Running, Role: LEADER
I20260806 00:44:10.271147 5594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e37cb5439c040998bbf5a5022d31b5b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46517 } }
I20260806 00:44:10.271670 5586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:10.272799 5595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e37cb5439c040998bbf5a5022d31b5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e37cb5439c040998bbf5a5022d31b5b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46517 } } }
I20260806 00:44:10.272827 5597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e37cb5439c040998bbf5a5022d31b5b. Latest consensus state: current_term: 1 leader_uuid: "1e37cb5439c040998bbf5a5022d31b5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e37cb5439c040998bbf5a5022d31b5b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 46517 } } }
I20260806 00:44:10.272958 5595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:10.272958 5597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:10.273334 5608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:10.276624 5608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:10.281698 5608 catalog_manager.cc:1347] Generated new cluster ID: f3ed00b29f37472f8f313e32a949e7e8
I20260806 00:44:10.281759 5608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:10.291985 5608 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:10.292800 5608 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:10.308663 5608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e37cb5439c040998bbf5a5022d31b5b: Generated new TSK 0
I20260806 00:44:10.309334 5608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:44:10.330987 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:10.325210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41564 (local address 127.29.84.190:46517)
0806 00:44:10.325342 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:10.325347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:10.325599 (+ 252us) server_negotiation.cc:407] Connection header received
0806 00:44:10.325742 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:10.325743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:10.325785 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:10.325864 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:44:10.326112 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.326732 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.328012 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.328226 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.328303 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:10.328617 (+ 314us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:44:10.330598 (+ 1981us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 00:44:10.330865 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41564: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 00:44:10.331053 5589 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41564: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 00:44:10.332628 5627 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:10.325157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:46517 (local address 127.0.0.1:41564)
0806 00:44:10.325510 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:10.325527 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:10.325635 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:10.325897 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:10.325900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:10.325907 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:44:10.326011 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:10.326017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.326848 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.326852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:10.327896 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:10.327906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.328355 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.328359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:10.328445 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:10.328471 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:44:10.330847 (+ 2376us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 00:44:10.332458 (+ 1611us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.84.190:46517: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":1415,"thread_start_us":111,"threads_started":1}
I20260806 00:44:10.333214 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 5483
E20260806 00:44:10.342640 5628 webserver.cc:533] Webserver: sslize failed:
E20260806 00:44:10.342736 5628 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T00:44:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T00:44:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:10Z Disabled control of system clock
I20260806 00:44:10.388031 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35889
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:35889
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/testchainca.cert with env {}
W20260806 00:44:10.493911 5639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:10.494138 5639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:10.494175 5639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:10.498046 5639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:10.498103 5639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:10.498121 5639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:10.498137 5639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:10.498206 5639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:44:10.498237 5639 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:44:10.498276 5639 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 00:44:10.503341 5639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:35889
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:10.504243 5639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:10.505168 5639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:44:10.513175 5639 server_base.cc:1034] running on GCE node
W20260806 00:44:10.513386 5645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:10.513438 5649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:10.513446 5646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:10.514114 5639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:10.515179 5639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:10.516364 5639 hybrid_clock.cc:648] HybridClock initialized: now 1785977050516346 us; error 37 us; skew 500 ppm
I20260806 00:44:10.518162 5639 webserver.cc:460] Webserver started at http://127.29.84.190:44963/ using document root <none> and password file <none>
I20260806 00:44:10.518710 5639 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:10.518805 5639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:10.519037 5639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:10.520959 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "696e46e5b0264dfdb38fcf40a4413f53"
format_stamp: "Formatted at 2026-08-06 00:44:10 on dist-test-slave-n326"
server_key: "5b3a796c90e935fd2ad5cc1d5cc0a87a"
server_key_iv: "503705bf3b599722775901b126e91d49"
server_key_version: "encryptionkey@0"
I20260806 00:44:10.521531 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "696e46e5b0264dfdb38fcf40a4413f53"
format_stamp: "Formatted at 2026-08-06 00:44:10 on dist-test-slave-n326"
server_key: "5b3a796c90e935fd2ad5cc1d5cc0a87a"
server_key_iv: "503705bf3b599722775901b126e91d49"
server_key_version: "encryptionkey@0"
I20260806 00:44:10.525053 5639 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:44:10.527318 5661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:10.528461 5639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:10.528602 5639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "696e46e5b0264dfdb38fcf40a4413f53"
format_stamp: "Formatted at 2026-08-06 00:44:10 on dist-test-slave-n326"
server_key: "5b3a796c90e935fd2ad5cc1d5cc0a87a"
server_key_iv: "503705bf3b599722775901b126e91d49"
server_key_version: "encryptionkey@0"
I20260806 00:44:10.528712 5639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:10.542074 5639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:10.542649 5639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:10.542851 5639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:10.549441 5639 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:35889
I20260806 00:44:10.549448 5737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:35889 every 8 connection(s)
I20260806 00:44:10.550283 5639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:10.552939 5738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:10.553264 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 5639
I20260806 00:44:10.553409 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:10.553723 30034 external_mini_cluster.cc:1428] Setting key 71105346bac31fd700ffe63776ea8250
I20260806 00:44:10.559028 5738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53: Bootstrap starting.
I20260806 00:44:10.561126 5738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:10.561894 5738 log.cc:824] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:10.562528 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:10.555039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55146 (local address 127.29.84.190:35889)
0806 00:44:10.555675 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:10.555691 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:10.555723 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:44:10.556355 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:10.556371 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:10.556731 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:10.557048 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:10.557461 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.558374 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.559374 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.559835 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.560017 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:10.560848 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:10.560853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:10.560865 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:10.561261 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:10.561316 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:10.561335 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:10.561478 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:10.561593 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 00:44:10.561829 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":137,"threads_started":1}
I20260806 00:44:10.563850 5738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53: No bootstrap required, opened a new log
I20260806 00:44:10.565356 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:44:10.567888 5738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696e46e5b0264dfdb38fcf40a4413f53" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35889 } }
I20260806 00:44:10.568080 5738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:10.568167 5738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696e46e5b0264dfdb38fcf40a4413f53, State: Initialized, Role: FOLLOWER
I20260806 00:44:10.568612 5738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696e46e5b0264dfdb38fcf40a4413f53" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35889 } }
I20260806 00:44:10.568753 5738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:10.568818 5738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:10.568918 5738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:10.569707 5738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696e46e5b0264dfdb38fcf40a4413f53" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35889 } }
I20260806 00:44:10.570025 5738 leader_election.cc:302] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 696e46e5b0264dfdb38fcf40a4413f53; no voters:
I20260806 00:44:10.570320 5738 leader_election.cc:288] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:10.570453 5743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:10.570673 5743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [term 1 LEADER]: Becoming Leader. State: Replica: 696e46e5b0264dfdb38fcf40a4413f53, State: Running, Role: LEADER
I20260806 00:44:10.570950 5743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696e46e5b0264dfdb38fcf40a4413f53" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35889 } }
I20260806 00:44:10.571595 5738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:10.572638 5744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "696e46e5b0264dfdb38fcf40a4413f53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696e46e5b0264dfdb38fcf40a4413f53" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35889 } } }
I20260806 00:44:10.572784 5744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:10.573110 5745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 696e46e5b0264dfdb38fcf40a4413f53. Latest consensus state: current_term: 1 leader_uuid: "696e46e5b0264dfdb38fcf40a4413f53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696e46e5b0264dfdb38fcf40a4413f53" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35889 } } }
I20260806 00:44:10.573202 5745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:10.573287 5754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:10.575837 5754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:10.580992 5754 catalog_manager.cc:1347] Generated new cluster ID: 2902e1e1b24f446098da9d4cdac8e5b3
I20260806 00:44:10.581053 5754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:10.588274 5754 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:10.589042 5754 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:10.593145 5754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 696e46e5b0264dfdb38fcf40a4413f53: Generated new TSK 0
I20260806 00:44:10.593703 5754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:44:10.688297 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:10.607410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.29.84.190:35889)
0806 00:44:10.607558 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:10.607563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:10.607705 (+ 142us) server_negotiation.cc:407] Connection header received
0806 00:44:10.607830 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:10.607833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:10.607886 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:10.607988 (+ 102us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:44:10.608192 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.609055 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.609883 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.610234 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.610338 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:10.610633 (+ 295us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:44:10.687807 (+ 77174us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 00:44:10.688143 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55150: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 00:44:10.688393 5741 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55150: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 00:44:10.688411 5775 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:10.607301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35889 (local address 127.0.0.1:55150)
0806 00:44:10.607614 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:10.607626 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:10.607714 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:10.607993 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:10.607995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:10.608001 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:44:10.608085 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:10.608089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.609160 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.609162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:10.609765 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:10.609772 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:10.610356 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:10.610359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:10.610438 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:10.610459 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:44:10.688045 (+ 77586us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 00:44:10.688223 (+ 178us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.84.190:35889: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":105,"threads_started":1}
I20260806 00:44:10.689261 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 5639
2026-08-06T00:44:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (337 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (121 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (119 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:10 dist-test-slave-n326 krb5kdc[5790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:10 dist-test-slave-n326 krb5kdc[5790](info): set up 2 sockets
Aug 06 00:44:10 dist-test-slave-n326 krb5kdc[5790](info): commencing operation
krb5kdc: starting...
W20260806 00:44:12.150239 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:12 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:44:12.271898 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:41251
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:41251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:12.376708 5806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:12.376915 5806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:12.376952 5806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:12.380798 5806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:12.380844 5806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:12.380862 5806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:12.380878 5806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:12.380918 5806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:12.385890 5806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:41251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:41251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:12.386747 5806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:12.387686 5806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:12.395793 5817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:12.395793 5820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:12.395867 5806 server_base.cc:1034] running on GCE node
W20260806 00:44:12.395792 5816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:12.396654 5806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:12.397608 5806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:12.398768 5806 hybrid_clock.cc:648] HybridClock initialized: now 1785977052398742 us; error 39 us; skew 500 ppm
Aug 06 00:44:12 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977052, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:12.401639 5806 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:44:12.402470 5806 webserver.cc:460] Webserver started at http://127.29.84.190:39671/ using document root <none> and password file <none>
I20260806 00:44:12.402990 5806 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:12.403101 5806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:12.403330 5806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:12.405184 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "89f3d1845aa048e9b3b0e5fcff55b097"
format_stamp: "Formatted at 2026-08-06 00:44:12 on dist-test-slave-n326"
server_key: "b18628320877d39e88b21eb33078b404"
server_key_iv: "5242919429d44481463a3892bbf16754"
server_key_version: "encryptionkey@0"
I20260806 00:44:12.405745 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "89f3d1845aa048e9b3b0e5fcff55b097"
format_stamp: "Formatted at 2026-08-06 00:44:12 on dist-test-slave-n326"
server_key: "b18628320877d39e88b21eb33078b404"
server_key_iv: "5242919429d44481463a3892bbf16754"
server_key_version: "encryptionkey@0"
I20260806 00:44:12.409255 5806 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:44:12.411478 5830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:12.412530 5806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:12.412674 5806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "89f3d1845aa048e9b3b0e5fcff55b097"
format_stamp: "Formatted at 2026-08-06 00:44:12 on dist-test-slave-n326"
server_key: "b18628320877d39e88b21eb33078b404"
server_key_iv: "5242919429d44481463a3892bbf16754"
server_key_version: "encryptionkey@0"
I20260806 00:44:12.412781 5806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:12.436096 5806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:12.439189 5806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:12.439380 5806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:12.446213 5806 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:41251
I20260806 00:44:12.446225 5902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:41251 every 8 connection(s)
I20260806 00:44:12.447105 5806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:12.449821 5903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:12.455066 5903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097: Bootstrap starting.
I20260806 00:44:12.457345 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 5806
I20260806 00:44:12.457479 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:12.457530 5903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:12.457804 30034 external_mini_cluster.cc:1428] Setting key 9bac0218225df9b4a29834991a529e2e
I20260806 00:44:12.458392 5903 log.cc:824] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:12.460228 5903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097: No bootstrap required, opened a new log
I20260806 00:44:12.464316 5903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f3d1845aa048e9b3b0e5fcff55b097" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41251 } }
I20260806 00:44:12.464567 5903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:12.464679 5903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f3d1845aa048e9b3b0e5fcff55b097, State: Initialized, Role: FOLLOWER
I20260806 00:44:12.465206 5903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f3d1845aa048e9b3b0e5fcff55b097" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41251 } }
I20260806 00:44:12.465380 5903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:12.465456 5903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:12.465566 5903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:12.466640 5903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f3d1845aa048e9b3b0e5fcff55b097" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41251 } }
Aug 06 00:44:12 dist-test-slave-n326 krb5kdc[5790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:12.467016 5903 leader_election.cc:302] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89f3d1845aa048e9b3b0e5fcff55b097; no voters:
I20260806 00:44:12.467363 5903 leader_election.cc:288] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:12.467494 5913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:12.467756 5913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [term 1 LEADER]: Becoming Leader. State: Replica: 89f3d1845aa048e9b3b0e5fcff55b097, State: Running, Role: LEADER
I20260806 00:44:12.468084 5913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f3d1845aa048e9b3b0e5fcff55b097" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41251 } }
I20260806 00:44:12.468571 5903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:12.469743 5915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89f3d1845aa048e9b3b0e5fcff55b097" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f3d1845aa048e9b3b0e5fcff55b097" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41251 } } }
I20260806 00:44:12.469882 5915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:12.470256 5928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:12.470214 5916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89f3d1845aa048e9b3b0e5fcff55b097. Latest consensus state: current_term: 1 leader_uuid: "89f3d1845aa048e9b3b0e5fcff55b097" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f3d1845aa048e9b3b0e5fcff55b097" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41251 } } }
I20260806 00:44:12.470467 5916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:12.473398 5928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:12.475864 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:12.458931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54006 (local address 127.29.84.190:41251)
0806 00:44:12.459380 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:12.459395 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:12.459428 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:44:12.460006 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:12.460022 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:12.460360 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:12.460622 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:12.461568 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.462556 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.463903 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.464368 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.464484 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:12.467524 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:12.467530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:12.467546 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:12.467578 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:12.472340 (+ 4762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:12.472855 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:12.472863 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:12.472868 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:12.473008 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:12.473336 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:12.473339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:12.473341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:12.474085 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:12.474301 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:12.474597 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 00:44:12.474892 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":152,"threads_started":1}
I20260806 00:44:12.478833 5928 catalog_manager.cc:1347] Generated new cluster ID: 5040ab7c4aaf47288a40bd4e2eaa8109
I20260806 00:44:12.478890 5928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:12.488003 5928 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:12.488936 5928 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:12.493825 5928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89f3d1845aa048e9b3b0e5fcff55b097: Generated new TSK 0
I20260806 00:44:12.494377 5928 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:44:12.527228 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:41251
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:12.635002 5952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:12.635272 5952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:12.635344 5952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:12.639173 5952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:12.639248 5952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:12.639335 5952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:12.644925 5952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:41251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:12.645900 5952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:12.646863 5952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:12.655762 5959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:12.655773 5963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:12.655776 5960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:12.656060 5952 server_base.cc:1034] running on GCE node
I20260806 00:44:12.656489 5952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:12.657063 5952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:12.658248 5952 hybrid_clock.cc:648] HybridClock initialized: now 1785977052658214 us; error 44 us; skew 500 ppm
Aug 06 00:44:12 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977052, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:12.663873 5952 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:44:12.664687 5952 webserver.cc:460] Webserver started at http://127.29.84.129:46419/ using document root <none> and password file <none>
I20260806 00:44:12.665210 5952 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:12.665302 5952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:12.665520 5952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:12.667493 5952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "9da264c743f84120b7ebdd607f774eaf"
format_stamp: "Formatted at 2026-08-06 00:44:12 on dist-test-slave-n326"
server_key: "b7495698257007ab658948081334e3c7"
server_key_iv: "c48a0f2bca427efccabd47a60e68e4f8"
server_key_version: "encryptionkey@0"
I20260806 00:44:12.668046 5952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "9da264c743f84120b7ebdd607f774eaf"
format_stamp: "Formatted at 2026-08-06 00:44:12 on dist-test-slave-n326"
server_key: "b7495698257007ab658948081334e3c7"
server_key_iv: "c48a0f2bca427efccabd47a60e68e4f8"
server_key_version: "encryptionkey@0"
I20260806 00:44:12.671789 5952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:44:12.674031 5973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:12.675179 5952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:12.675302 5952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "9da264c743f84120b7ebdd607f774eaf"
format_stamp: "Formatted at 2026-08-06 00:44:12 on dist-test-slave-n326"
server_key: "b7495698257007ab658948081334e3c7"
server_key_iv: "c48a0f2bca427efccabd47a60e68e4f8"
server_key_version: "encryptionkey@0"
I20260806 00:44:12.675412 5952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:12.693246 5952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:12.695829 5952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:12.696019 5952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:12.696580 5952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:12.697345 5952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:12.697419 5952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:12.697484 5952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:12.697532 5952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:12.706095 5952 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:33547
I20260806 00:44:12.706175 6138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:33547 every 8 connection(s)
I20260806 00:44:12.706971 5952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:12.712862 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 5952
I20260806 00:44:12.713009 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:12.713291 30034 external_mini_cluster.cc:1428] Setting key 9d637cb20f5a2d814fa36222391ec9ed
Aug 06 00:44:12 dist-test-slave-n326 krb5kdc[5790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977052, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:12.722154 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:12.708742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:41007 (local address 127.29.84.190:41251)
0806 00:44:12.708870 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:12.708874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:12.709629 (+ 755us) server_negotiation.cc:407] Connection header received
0806 00:44:12.711408 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:12.711410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:12.711457 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:12.711532 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:12.712650 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.713372 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.714772 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.714997 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.715129 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:12.718226 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:12.718229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:12.718231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:12.718253 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:12.719846 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:12.720486 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:12.720489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:12.720490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:12.720562 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:12.721009 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:12.721011 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:12.721012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:12.721192 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:12.721265 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:12.721931 (+ 666us) server_negotiation.cc:299] Negotiation successful
0806 00:44:12.722031 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:44:12.723299 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:12.709069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:41251 (local address 127.29.84.129:41007)
0806 00:44:12.709488 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:12.709527 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:12.711199 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:12.711694 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:12.711700 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:12.712023 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:12.712482 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:12.712494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.713518 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.713522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:12.714656 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:12.714671 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.715178 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.715185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:12.715402 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:12.716033 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:12.716057 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:12.718061 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:12.719993 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:12.719999 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:12.720013 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:12.720388 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:12.720658 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:12.720663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:12.720668 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:12.720904 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:12.721298 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:12.721306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:12.721671 (+ 365us) client_negotiation.cc:769] Sending connection context
0806 00:44:12.721906 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 00:44:12.722253 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":129,"threads_started":1}
I20260806 00:44:12.724488 6140 heartbeater.cc:347] Connected to a master server at 127.29.84.190:41251
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:44:12.724727 6140 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:12.725226 6140 heartbeater.cc:511] Master 127.29.84.190:41251 requested a full tablet report, sending...
I20260806 00:44:12.726586 5857 ts_manager.cc:194] Registered new tserver with Master: 9da264c743f84120b7ebdd607f774eaf (127.29.84.129:33547)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
I20260806 00:44:12.727924 5857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:41007
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:44:12.742035 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:41251
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:12.847910 6149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:12.848124 6149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:12.848163 6149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:12.852136 6149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:12.852195 6149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:12.852268 6149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:12.857308 6149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:41251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:12.858158 6149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:12.859045 6149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:12.867875 6155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:12.867910 6158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:12.867966 6161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:12.868182 6149 server_base.cc:1034] running on GCE node
I20260806 00:44:12.868554 6149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:12.869151 6149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:12.870332 6149 hybrid_clock.cc:648] HybridClock initialized: now 1785977052870316 us; error 34 us; skew 500 ppm
Aug 06 00:44:12 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977052, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:12.872959 6149 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:44:12.873734 6149 webserver.cc:460] Webserver started at http://127.29.84.130:36213/ using document root <none> and password file <none>
I20260806 00:44:12.874191 6149 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:12.874246 6149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:12.874393 6149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:12.876184 6149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "4cdbd6b064f346d29240208e72a3cd5b"
format_stamp: "Formatted at 2026-08-06 00:44:12 on dist-test-slave-n326"
server_key: "559681267ddab5e93d6229a82559a690"
server_key_iv: "b791f03be63ea5712bf7f11c067450c6"
server_key_version: "encryptionkey@0"
I20260806 00:44:12.876667 6149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "4cdbd6b064f346d29240208e72a3cd5b"
format_stamp: "Formatted at 2026-08-06 00:44:12 on dist-test-slave-n326"
server_key: "559681267ddab5e93d6229a82559a690"
server_key_iv: "b791f03be63ea5712bf7f11c067450c6"
server_key_version: "encryptionkey@0"
I20260806 00:44:12.880106 6149 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:44:12.882293 6174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:12.883471 6149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:12.883560 6149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "4cdbd6b064f346d29240208e72a3cd5b"
format_stamp: "Formatted at 2026-08-06 00:44:12 on dist-test-slave-n326"
server_key: "559681267ddab5e93d6229a82559a690"
server_key_iv: "b791f03be63ea5712bf7f11c067450c6"
server_key_version: "encryptionkey@0"
I20260806 00:44:12.883635 6149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:12.898281 6149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:12.900812 6149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:12.900946 6149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:12.901400 6149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:12.902117 6149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:12.902165 6149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:12.902200 6149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:12.902222 6149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:12.911245 6149 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:34509
I20260806 00:44:12.911336 6337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:34509 every 8 connection(s)
I20260806 00:44:12.912034 6149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:12.918099 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 6149
I20260806 00:44:12.918226 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:12.918497 30034 external_mini_cluster.cc:1428] Setting key 7fbcab0c57f09fc3174803820f738cba
Aug 06 00:44:12 dist-test-slave-n326 krb5kdc[5790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977052, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:12.925313 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:12.913697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:38117 (local address 127.29.84.190:41251)
0806 00:44:12.913835 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:12.913839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:12.914600 (+ 761us) server_negotiation.cc:407] Connection header received
0806 00:44:12.915955 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:12.915958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:12.916003 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:12.916075 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:12.917151 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.917688 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.918932 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.919234 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.919324 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:12.921728 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:12.921739 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:12.921741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:12.921763 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:12.923304 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:12.923827 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:12.923835 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:12.923836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:12.923909 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:12.924244 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:12.924249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:12.924251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:12.924428 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:12.924539 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:12.925030 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 00:44:12.925147 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:44:12.926072 6340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:12.914016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:41251 (local address 127.29.84.130:38117)
0806 00:44:12.914453 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:12.914495 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:12.915771 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:12.916260 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:12.916272 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:12.916611 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:12.917025 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:12.917037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.917795 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.917798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:12.918790 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:12.918800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:12.919388 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:12.919392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:12.919596 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:12.920147 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:12.920161 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:12.921563 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:12.923436 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:12.923444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:12.923456 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:12.923709 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:12.924004 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:12.924007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:12.924009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:12.924145 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:12.924538 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:12.924544 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:12.924765 (+ 221us) client_negotiation.cc:769] Sending connection context
0806 00:44:12.924979 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 00:44:12.925267 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260806 00:44:12.927297 6338 heartbeater.cc:347] Connected to a master server at 127.29.84.190:41251
I20260806 00:44:12.927557 6338 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:12.927994 6338 heartbeater.cc:511] Master 127.29.84.190:41251 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:44:12.929034 5857 ts_manager.cc:194] Registered new tserver with Master: 4cdbd6b064f346d29240208e72a3cd5b (127.29.84.130:34509)
I20260806 00:44:12.929577 5857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:38117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:44:12.945835 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:41251
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:13.052667 6344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:13.052865 6344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:13.052902 6344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:13.056495 6344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:13.056540 6344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:13.056608 6344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:13.061343 6344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:41251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:13.062179 6344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:13.063055 6344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:13.071746 6355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:13.071751 6351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:13.071751 6352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:13.071934 6344 server_base.cc:1034] running on GCE node
I20260806 00:44:13.072386 6344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:13.072939 6344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:13.074097 6344 hybrid_clock.cc:648] HybridClock initialized: now 1785977053074074 us; error 40 us; skew 500 ppm
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:13.076758 6344 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:44:13.077575 6344 webserver.cc:460] Webserver started at http://127.29.84.131:33479/ using document root <none> and password file <none>
I20260806 00:44:13.078081 6344 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:13.078171 6344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:13.078389 6344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:13.080336 6344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "87d25a8c368f43298fb1e83d83ffd43f"
format_stamp: "Formatted at 2026-08-06 00:44:13 on dist-test-slave-n326"
server_key: "bf061c68da807bd20471842173a2a5fd"
server_key_iv: "9063069497864bb249d2fb3013cf6e8f"
server_key_version: "encryptionkey@0"
I20260806 00:44:13.080886 6344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "87d25a8c368f43298fb1e83d83ffd43f"
format_stamp: "Formatted at 2026-08-06 00:44:13 on dist-test-slave-n326"
server_key: "bf061c68da807bd20471842173a2a5fd"
server_key_iv: "9063069497864bb249d2fb3013cf6e8f"
server_key_version: "encryptionkey@0"
I20260806 00:44:13.084234 6344 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:13.086344 6366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:13.087543 6344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:44:13.087682 6344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "87d25a8c368f43298fb1e83d83ffd43f"
format_stamp: "Formatted at 2026-08-06 00:44:13 on dist-test-slave-n326"
server_key: "bf061c68da807bd20471842173a2a5fd"
server_key_iv: "9063069497864bb249d2fb3013cf6e8f"
server_key_version: "encryptionkey@0"
I20260806 00:44:13.087790 6344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:13.100296 6344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:13.102967 6344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:13.103174 6344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:13.103689 6344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:13.104421 6344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:13.104468 6344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:13.104529 6344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:13.104565 6344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:13.112990 6344 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:38825
I20260806 00:44:13.113016 6541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:38825 every 8 connection(s)
I20260806 00:44:13.113742 6344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:13.121623 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 6344
I20260806 00:44:13.121749 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:13.122067 30034 external_mini_cluster.cc:1428] Setting key 952c3642f0aa51f82e5bae0b59888fd7
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:13.126394 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:13.115295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:53021 (local address 127.29.84.190:41251)
0806 00:44:13.115467 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:13.115470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:13.116191 (+ 721us) server_negotiation.cc:407] Connection header received
0806 00:44:13.117555 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:13.117558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:13.117631 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:13.117728 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:13.118778 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.119341 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.120403 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.120619 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.120721 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:13.123264 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:13.123268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:13.123272 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:13.123302 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:13.124739 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:13.125204 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:13.125207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:13.125208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:13.125276 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:13.125548 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:13.125550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:13.125551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:13.125734 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:13.125807 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:13.126197 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 00:44:13.126287 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 00:44:13.127143 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:13.115670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:41251 (local address 127.29.84.131:53021)
0806 00:44:13.116057 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:13.116101 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:13.117337 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:13.117892 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:13.117899 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:13.118220 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:13.118628 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:13.118640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.119461 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.119465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:13.120276 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:13.120286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.120768 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.120773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:13.120929 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:13.121397 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:13.121431 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:13.123089 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:13.124840 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:13.124844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:13.124854 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:13.125104 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:13.125353 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:13.125355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:13.125356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:13.125473 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:13.125820 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:13.125824 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:13.126016 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 00:44:13.126151 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 00:44:13.126354 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":105,"threads_started":1}
I20260806 00:44:13.128113 6542 heartbeater.cc:347] Connected to a master server at 127.29.84.190:41251
I20260806 00:44:13.128329 6542 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:13.128767 6542 heartbeater.cc:511] Master 127.29.84.190:41251 requested a full tablet report, sending...
I20260806 00:44:13.129667 5857 ts_manager.cc:194] Registered new tserver with Master: 87d25a8c368f43298fb1e83d83ffd43f (127.29.84.131:38825)
I20260806 00:44:13.130183 5857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:53021
I20260806 00:44:13.135591 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/test-admin.keytab.
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:13.151213 30034 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 00:44:13.153687 6554 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:13.152432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.29.84.129:33547)
0806 00:44:13.152737 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:13.152747 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:13.152767 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:44:13.152810 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:13.152814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:13.152953 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:13.153070 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:13.153521 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38346: BlockingRecv error: recv got EOF from 127.0.0.1:38346 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":114,"threads_started":1}
W20260806 00:44:13.153697 30034 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5cc' of type FILE
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:13.154697 30034 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:13.155652 30034 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:44:13.164203 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:13.156387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.29.84.129:33547)
0806 00:44:13.156522 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:13.156525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:13.156538 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:44:13.156667 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:13.156669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:13.156702 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:13.156763 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:13.157212 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.157977 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.158749 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.159109 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.159196 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:13.161200 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:13.161209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:13.161218 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:13.161239 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:13.162690 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:13.163057 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:13.163061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:13.163111 (+ 50us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:13.163190 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:13.163406 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:13.163408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:13.163409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:13.163601 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:13.163736 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:13.163880 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 00:44:13.164060 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 00:44:13.167593 6554 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:13.166530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38352 (local address 127.29.84.129:33547)
0806 00:44:13.166632 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:13.166636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:13.166748 (+ 112us) server_negotiation.cc:407] Connection header received
0806 00:44:13.166823 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:13.166825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:13.166854 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:13.166912 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:13.167485 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38352: BlockingRecv error: recv got EOF from 127.0.0.1:38352 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 00:44:13.167850 30034 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5cc)
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:13.168749 30034 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:13.169660 30034 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:44:13.177796 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:13.170408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.29.84.129:33547)
0806 00:44:13.170519 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:13.170523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:13.170631 (+ 108us) server_negotiation.cc:407] Connection header received
0806 00:44:13.170742 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:13.170744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:13.170778 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:13.170863 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:13.171405 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.171875 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.172656 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.172901 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.173018 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:13.174997 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:13.175001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:13.175003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:13.175032 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:13.176229 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:13.176643 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:13.176646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:13.176648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:13.176736 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:13.177011 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:13.177014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:13.177015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:13.177172 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:13.177297 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:13.177556 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 00:44:13.177659 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 00:44:13.180207 6554 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:13.179338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.29.84.129:33547)
0806 00:44:13.179446 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:13.179450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:13.179512 (+ 62us) server_negotiation.cc:407] Connection header received
0806 00:44:13.179635 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:13.179638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:13.179670 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:13.179759 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:13.180123 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38368: BlockingRecv error: recv got EOF from 127.0.0.1:38368 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 00:44:13.180328 30034 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977020829091-30034-0/krb5kdc/krb5cc' of type FILE
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:13.181185 30034 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:13.182053 30034 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[5790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:44:13.189356 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:13.182765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.29.84.129:33547)
0806 00:44:13.182888 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:13.182891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:13.182903 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:44:13.182997 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:13.182999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:13.183028 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:13.183142 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:13.183537 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.183946 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.184617 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:13.184802 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:13.184869 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:13.186831 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:13.186834 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:13.186836 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:13.186855 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:13.188069 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:13.188426 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:13.188428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:13.188429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:13.188497 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:13.188731 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:13.188733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:13.188734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:13.188854 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:13.188937 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:13.189111 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 00:44:13.189212 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:44:13.190251 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 5952
I20260806 00:44:13.204947 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 6149
I20260806 00:44:13.217151 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 6344
I20260806 00:44:13.228901 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 5806
2026-08-06T00:44:13Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2296 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[6569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[6569](info): set up 2 sockets
Aug 06 00:44:13 dist-test-slave-n326 krb5kdc[6569](info): commencing operation
krb5kdc: starting...
W20260806 00:44:14.428097 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:14 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.190.keytab.
Entry for principal oryx/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.190.keytab.
I20260806 00:44:14.545408 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:44055
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.190.keytab
--principal=oryx/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:44055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:14.651471 6594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:14.651682 6594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:14.651718 6594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:14.655604 6594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:14.655651 6594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:14.655670 6594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:14.655686 6594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:14.655740 6594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:14.660722 6594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:44055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.190.keytab
--principal=oryx/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:44055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:14.661586 6594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:14.662505 6594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:14.670204 6602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:14.670195 6609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:14.670334 6594 server_base.cc:1034] running on GCE node
W20260806 00:44:14.670248 6604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:14.670986 6594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:14.671959 6594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:14.673167 6594 hybrid_clock.cc:648] HybridClock initialized: now 1785977054673148 us; error 31 us; skew 500 ppm
Aug 06 00:44:14 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977054, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:14.675773 6594 init.cc:377] Logged in from keytab as oryx/127.29.84.190@KRBTEST.COM (short username oryx)
I20260806 00:44:14.676496 6594 webserver.cc:460] Webserver started at http://127.29.84.190:33565/ using document root <none> and password file <none>
I20260806 00:44:14.676932 6594 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:14.676986 6594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:14.677130 6594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:14.678926 6594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "56329dd2abc5442fbb981a835e5df66a"
format_stamp: "Formatted at 2026-08-06 00:44:14 on dist-test-slave-n326"
server_key: "1b3e1dd2b4633cd4b4a833be8c781c31"
server_key_iv: "2215c1cd9eb535effdf9fe188dde969f"
server_key_version: "encryptionkey@0"
I20260806 00:44:14.679483 6594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "56329dd2abc5442fbb981a835e5df66a"
format_stamp: "Formatted at 2026-08-06 00:44:14 on dist-test-slave-n326"
server_key: "1b3e1dd2b4633cd4b4a833be8c781c31"
server_key_iv: "2215c1cd9eb535effdf9fe188dde969f"
server_key_version: "encryptionkey@0"
I20260806 00:44:14.682854 6594 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:44:14.685156 6619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:14.686307 6594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:14.686448 6594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "56329dd2abc5442fbb981a835e5df66a"
format_stamp: "Formatted at 2026-08-06 00:44:14 on dist-test-slave-n326"
server_key: "1b3e1dd2b4633cd4b4a833be8c781c31"
server_key_iv: "2215c1cd9eb535effdf9fe188dde969f"
server_key_version: "encryptionkey@0"
I20260806 00:44:14.686550 6594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:14.716002 6594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:14.718670 6594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:14.718880 6594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:14.725597 6594 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:44055
I20260806 00:44:14.725878 6701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:44055 every 8 connection(s)
I20260806 00:44:14.726552 6594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:14.729220 6702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:14.731288 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 6594
I20260806 00:44:14.731437 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:14.731761 30034 external_mini_cluster.cc:1428] Setting key 311437f89e4916fe9e821994a652361b
I20260806 00:44:14.735857 6702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: Bootstrap starting.
I20260806 00:44:14.738303 6702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:14.739241 6702 log.cc:824] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:44:14 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:14.741159 6702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: No bootstrap required, opened a new log
I20260806 00:44:14.745419 6702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } }
I20260806 00:44:14.745671 6702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:14.745769 6702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56329dd2abc5442fbb981a835e5df66a, State: Initialized, Role: FOLLOWER
I20260806 00:44:14.746212 6702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } }
I20260806 00:44:14.746377 6702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:14.746474 6702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:14.746572 6702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:14.747718 6702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } }
I20260806 00:44:14.748063 6702 leader_election.cc:302] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56329dd2abc5442fbb981a835e5df66a; no voters:
I20260806 00:44:14.748385 6702 leader_election.cc:288] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:14.748528 6709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:14.748767 6709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 1 LEADER]: Becoming Leader. State: Replica: 56329dd2abc5442fbb981a835e5df66a, State: Running, Role: LEADER
I20260806 00:44:14.749060 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:14.733081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.29.84.190:44055)
0806 00:44:14.733468 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:14.733489 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:14.733522 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:44:14.734128 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:14.734144 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:14.734435 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:14.734687 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:14.735601 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:14.736624 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:14.737773 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:14.738214 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:14.738326 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:14.741070 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:14.741075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:14.741086 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:14.741110 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:14.745894 (+ 4784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:14.746356 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:14.746362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:14.746367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:14.746537 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:14.746851 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:14.746854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:14.746856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:14.747358 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:14.747572 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:14.747795 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 00:44:14.748078 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":134,"threads_started":1}
I20260806 00:44:14.749105 6709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } }
I20260806 00:44:14.749643 6702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:14.751087 6711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56329dd2abc5442fbb981a835e5df66a. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } }
I20260806 00:44:14.751077 6713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } }
I20260806 00:44:14.751221 6711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:14.751221 6713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
W20260806 00:44:14.754458 6733 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:44:14.754537 6733 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:44:14.754599 6727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:14.755327 6727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:14.760481 6727 catalog_manager.cc:1347] Generated new cluster ID: 4f5b0b9da0a54bf6bca0da60e0e9fabc
I20260806 00:44:14.760541 6727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:14.768204 6727 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:14.769244 6727 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:14.774072 6727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: Generated new TSK 0
I20260806 00:44:14.774657 6727 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab.
Entry for principal oryx/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab.
I20260806 00:44:14.802475 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:44055
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab
--principal=oryx/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:14.908789 6739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:14.909051 6739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:14.909129 6739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:14.912976 6739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:14.913050 6739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:14.913126 6739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:14.917887 6739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab
--principal=oryx/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:44055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:14.918752 6739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:14.919648 6739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:14.928471 6745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:14.928536 6744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:14.928591 6739 server_base.cc:1034] running on GCE node
W20260806 00:44:14.928515 6749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:14.929049 6739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:14.929628 6739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:14.930785 6739 hybrid_clock.cc:648] HybridClock initialized: now 1785977054930756 us; error 41 us; skew 500 ppm
Aug 06 00:44:14 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977054, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:14.933377 6739 init.cc:377] Logged in from keytab as oryx/127.29.84.129@KRBTEST.COM (short username oryx)
I20260806 00:44:14.934136 6739 webserver.cc:460] Webserver started at http://127.29.84.129:41849/ using document root <none> and password file <none>
I20260806 00:44:14.934659 6739 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:14.934749 6739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:14.934976 6739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:14.936844 6739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "ca9e4be9fc804074a3d0d02d481b3b38"
format_stamp: "Formatted at 2026-08-06 00:44:14 on dist-test-slave-n326"
server_key: "66e999fe021aca821bb1918b071630fe"
server_key_iv: "57047847bee9a2d0add478664e7df966"
server_key_version: "encryptionkey@0"
I20260806 00:44:14.937388 6739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "ca9e4be9fc804074a3d0d02d481b3b38"
format_stamp: "Formatted at 2026-08-06 00:44:14 on dist-test-slave-n326"
server_key: "66e999fe021aca821bb1918b071630fe"
server_key_iv: "57047847bee9a2d0add478664e7df966"
server_key_version: "encryptionkey@0"
I20260806 00:44:14.940768 6739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:44:14.942885 6762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:14.944087 6739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:44:14.944211 6739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "ca9e4be9fc804074a3d0d02d481b3b38"
format_stamp: "Formatted at 2026-08-06 00:44:14 on dist-test-slave-n326"
server_key: "66e999fe021aca821bb1918b071630fe"
server_key_iv: "57047847bee9a2d0add478664e7df966"
server_key_version: "encryptionkey@0"
I20260806 00:44:14.944306 6739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:14.963467 6739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:14.965798 6739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:14.965986 6739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:14.966495 6739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:14.967298 6739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:14.967344 6739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:14.967406 6739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:14.967438 6739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:14.976028 6739 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:33915
I20260806 00:44:14.976065 6933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:33915 every 8 connection(s)
I20260806 00:44:14.976778 6739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:14.978104 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 6739
I20260806 00:44:14.978246 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:14.978609 30034 external_mini_cluster.cc:1428] Setting key 4cc3b3d42830e0a8319bbba12d3c1ad4
Aug 06 00:44:14 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977054, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:44:14.990749 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:14.978357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:53681 (local address 127.29.84.190:44055)
0806 00:44:14.978512 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:14.978518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:14.979291 (+ 773us) server_negotiation.cc:407] Connection header received
0806 00:44:14.980623 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:14.980629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:14.980690 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:14.980783 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:14.981918 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:14.982454 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:14.984172 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:14.984413 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:14.984490 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:14.987248 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:14.987251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:14.987253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:14.987274 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:14.988706 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:14.989272 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:14.989274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:14.989275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:14.989341 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:14.989739 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:14.989742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:14.989743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:14.989906 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:14.989983 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:14.990488 (+ 505us) server_negotiation.cc:299] Negotiation successful
0806 00:44:14.990623 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.130@KRBTEST.COM" created.
I20260806 00:44:14.991631 6939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:14.978676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.129:53681)
0806 00:44:14.979153 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:14.979201 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:14.980426 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:14.980928 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:14.980934 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:14.981277 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:14.981733 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:14.981756 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:14.982593 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:14.982599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:14.984039 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:14.984054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:14.984569 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:14.984576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:14.984796 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:14.985357 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:14.985381 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:14.987007 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:14.988839 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:14.988844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:14.988856 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:14.989128 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:14.989444 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:14.989447 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:14.989449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:14.989593 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:14.990027 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:14.990034 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:14.990239 (+ 205us) client_negotiation.cc:769] Sending connection context
0806 00:44:14.990429 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 00:44:14.990647 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":147,"threads_started":1}
I20260806 00:44:14.992784 6934 heartbeater.cc:347] Connected to a master server at 127.29.84.190:44055
I20260806 00:44:14.993009 6934 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:14.993556 6934 heartbeater.cc:511] Master 127.29.84.190:44055 requested a full tablet report, sending...
I20260806 00:44:14.994974 6640 ts_manager.cc:194] Registered new tserver with Master: ca9e4be9fc804074a3d0d02d481b3b38 (127.29.84.129:33915)
I20260806 00:44:14.996095 6640 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:53681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab.
Entry for principal oryx/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab.
I20260806 00:44:15.006387 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:44055
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab
--principal=oryx/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:15.112576 6945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:15.112771 6945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:15.112836 6945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:15.116626 6945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:15.116670 6945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:15.116722 6945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:15.121577 6945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab
--principal=oryx/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:44055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:15.122403 6945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:15.123337 6945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:15.131739 6954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:15.131775 6957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:15.131785 6953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:15.132035 6945 server_base.cc:1034] running on GCE node
I20260806 00:44:15.132386 6945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:15.132943 6945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:15.134125 6945 hybrid_clock.cc:648] HybridClock initialized: now 1785977055134113 us; error 32 us; skew 500 ppm
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:15.136653 6945 init.cc:377] Logged in from keytab as oryx/127.29.84.130@KRBTEST.COM (short username oryx)
I20260806 00:44:15.137461 6945 webserver.cc:460] Webserver started at http://127.29.84.130:39869/ using document root <none> and password file <none>
I20260806 00:44:15.137974 6945 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:15.138067 6945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:15.138288 6945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:15.140123 6945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "be43b3654b7a495f8ab51fd621037e66"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0ca5dda42e2dae7d736195310bd715da"
server_key_iv: "d2c49c70a37a09ca624945434647610d"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.140638 6945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "be43b3654b7a495f8ab51fd621037e66"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0ca5dda42e2dae7d736195310bd715da"
server_key_iv: "d2c49c70a37a09ca624945434647610d"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.144052 6945 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:15.146242 6967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.147467 6945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:15.147606 6945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "be43b3654b7a495f8ab51fd621037e66"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0ca5dda42e2dae7d736195310bd715da"
server_key_iv: "d2c49c70a37a09ca624945434647610d"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.147709 6945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:15.167425 6945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:15.169929 6945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:15.170114 6945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:15.170629 6945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:15.171433 6945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:15.171504 6945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.171610 6945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:15.171664 6945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.180403 6945 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:35959
I20260806 00:44:15.180454 7164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:35959 every 8 connection(s)
I20260806 00:44:15.181218 6945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:15.182098 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 6945
I20260806 00:44:15.182230 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:15.182551 30034 external_mini_cluster.cc:1428] Setting key 268ff78e04078457594bbf1b21fd3ff0
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.29.84.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.131@KRBTEST.COM" created.
I20260806 00:44:15.196596 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.183002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:57585 (local address 127.29.84.190:44055)
0806 00:44:15.183171 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:15.183178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:15.184089 (+ 911us) server_negotiation.cc:407] Connection header received
0806 00:44:15.186079 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:15.186082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:15.186153 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:15.186251 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:15.187490 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.188084 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.189913 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.190187 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.190312 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.193104 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:15.193107 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:15.193109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:15.193131 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:15.194526 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.195026 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.195029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.195030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.195156 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.195452 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.195454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.195455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.195686 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:15.195842 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:15.196324 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 00:44:15.196440 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 00:44:15.197438 7167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.183426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.130:57585)
0806 00:44:15.183914 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:15.183973 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:15.185851 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:15.186459 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:15.186466 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:15.186800 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:15.187351 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.187364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.188224 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.188231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.189790 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.189805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.190396 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.190404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.190638 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.191326 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:15.191345 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:15.192936 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:15.194655 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.194659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.194669 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.194930 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.195238 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.195240 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.195241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.195364 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.195837 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:15.195841 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:15.196056 (+ 215us) client_negotiation.cc:769] Sending connection context
0806 00:44:15.196191 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 00:44:15.196490 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":124,"threads_started":1}
I20260806 00:44:15.198716 7165 heartbeater.cc:347] Connected to a master server at 127.29.84.190:44055
I20260806 00:44:15.198956 7165 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:15.199430 7165 heartbeater.cc:511] Master 127.29.84.190:44055 requested a full tablet report, sending...
I20260806 00:44:15.200426 6640 ts_manager.cc:194] Registered new tserver with Master: be43b3654b7a495f8ab51fd621037e66 (127.29.84.130:35959)
I20260806 00:44:15.200979 6640 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:57585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab.
Entry for principal oryx/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab.
I20260806 00:44:15.213395 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:44055
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab
--principal=oryx/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:15.319720 7172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:15.319922 7172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:15.319962 7172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:15.323722 7172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:15.323763 7172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:15.323817 7172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:15.328679 7172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab
--principal=oryx/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:44055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:15.329434 7172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:15.330250 7172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:15.338853 7178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:15.338881 7179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:15.338848 7181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:15.339218 7172 server_base.cc:1034] running on GCE node
I20260806 00:44:15.339574 7172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:15.340121 7172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:15.341316 7172 hybrid_clock.cc:648] HybridClock initialized: now 1785977055341302 us; error 37 us; skew 500 ppm
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:15.343945 7172 init.cc:377] Logged in from keytab as oryx/127.29.84.131@KRBTEST.COM (short username oryx)
I20260806 00:44:15.344677 7172 webserver.cc:460] Webserver started at http://127.29.84.131:46261/ using document root <none> and password file <none>
I20260806 00:44:15.345144 7172 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:15.345201 7172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:15.345350 7172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:15.347219 7172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "2b9c210f183840ec93c1270003921c37"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0bfae76ef078273d8d8f157457d09b21"
server_key_iv: "bdff71b3ef859d426a693abc10d32ae5"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.347813 7172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "2b9c210f183840ec93c1270003921c37"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0bfae76ef078273d8d8f157457d09b21"
server_key_iv: "bdff71b3ef859d426a693abc10d32ae5"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.351693 7172 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 00:44:15.353986 7193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.355163 7172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:15.355278 7172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "2b9c210f183840ec93c1270003921c37"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0bfae76ef078273d8d8f157457d09b21"
server_key_iv: "bdff71b3ef859d426a693abc10d32ae5"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.355387 7172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:15.386879 7172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:15.389822 7172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:15.390022 7172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:15.390609 7172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:15.391374 7172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:15.391454 7172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.391527 7172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:15.391571 7172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.399922 7172 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:33215
I20260806 00:44:15.399945 7376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:33215 every 8 connection(s)
I20260806 00:44:15.400698 7172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:15.409042 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 7172
I20260806 00:44:15.409183 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:15.409482 30034 external_mini_cluster.cc:1428] Setting key 21d0cd44da520d17a7a53f5e7dfab10b
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:15.413620 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.402368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:38527 (local address 127.29.84.190:44055)
0806 00:44:15.402500 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:15.402504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:15.403274 (+ 770us) server_negotiation.cc:407] Connection header received
0806 00:44:15.404591 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:15.404594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:15.404653 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:15.404757 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:15.405816 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.406326 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.407447 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.407686 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.407807 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.410330 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:15.410334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:15.410336 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:15.410363 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:15.411945 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.412401 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.412404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.412405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.412474 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.412765 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.412767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.412768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.412923 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:15.412995 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:15.413414 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 00:44:15.413499 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:44:15.414310 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.402663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.131:38527)
0806 00:44:15.403126 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:15.403175 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:15.404377 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:15.404917 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:15.404923 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:15.405253 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:15.405679 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.405691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.406449 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.406452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.407329 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.407338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.407843 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.407847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.408006 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.408524 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:15.408546 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:15.410172 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:15.412047 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.412051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.412061 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.412316 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.412557 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.412559 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.412560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.412686 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.413009 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:15.413013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:15.413216 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 00:44:15.413366 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 00:44:15.413574 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":117,"threads_started":1}
I20260806 00:44:15.415366 7377 heartbeater.cc:347] Connected to a master server at 127.29.84.190:44055
I20260806 00:44:15.415545 7377 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:15.415918 7377 heartbeater.cc:511] Master 127.29.84.190:44055 requested a full tablet report, sending...
I20260806 00:44:15.416767 6640 ts_manager.cc:194] Registered new tserver with Master: 2b9c210f183840ec93c1270003921c37 (127.29.84.131:33215)
I20260806 00:44:15.417237 6640 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:38527
I20260806 00:44:15.423113 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.84.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.189.keytab.
Entry for principal oryx/127.29.84.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.189.keytab.
I20260806 00:44:15.450201 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.189:43869
--webserver_interface=127.29.84.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.189.keytab
--principal=oryx/127.29.84.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:44055,127.29.84.189:43869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:15.558492 7388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:15.558718 7388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:15.558756 7388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:15.562783 7388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:15.562834 7388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:15.562853 7388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:15.562870 7388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:15.562885 7388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:15.567826 7388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:44055,127.29.84.189:43869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.189.keytab
--principal=oryx/127.29.84.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.189:43869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.84.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
W20260806 00:44:15.567986 7388 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.84.190:44055,127.29.84.189:43869'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 00:44:15.568693 7388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:15.569538 7388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:15.577540 7400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:15.577545 7395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:15.577546 7396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:15.577795 7388 server_base.cc:1034] running on GCE node
I20260806 00:44:15.578325 7388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:15.579324 7388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:15.580508 7388 hybrid_clock.cc:648] HybridClock initialized: now 1785977055580495 us; error 27 us; skew 500 ppm
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:15.583122 7388 init.cc:377] Logged in from keytab as oryx/127.29.84.189@KRBTEST.COM (short username oryx)
I20260806 00:44:15.583894 7388 webserver.cc:460] Webserver started at http://127.29.84.189:42337/ using document root <none> and password file <none>
I20260806 00:44:15.584403 7388 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:15.584491 7388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:15.584721 7388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:15.586598 7388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data/instance:
uuid: "5d8566684ab940cb804ba536f479df5e"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0f1806afaa0d74466a9737a11b099e9a"
server_key_iv: "2a85a28615c4b1a18a0a2536278df657"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.587200 7388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal/instance:
uuid: "5d8566684ab940cb804ba536f479df5e"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0f1806afaa0d74466a9737a11b099e9a"
server_key_iv: "2a85a28615c4b1a18a0a2536278df657"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.590762 7388 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:44:15.593027 7411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.594130 7388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:15.594264 7388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal
uuid: "5d8566684ab940cb804ba536f479df5e"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0f1806afaa0d74466a9737a11b099e9a"
server_key_iv: "2a85a28615c4b1a18a0a2536278df657"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.594364 7388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:15.622116 7388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:15.624755 7388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:15.624959 7388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:15.631771 7388 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.189:43869
I20260806 00:44:15.632042 7482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.189:43869 every 8 connection(s)
I20260806 00:44:15.632707 7388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
I20260806 00:44:15.635298 7483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:15.636168 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 7388
I20260806 00:44:15.639518 7483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.84.189@KRBTEST.COM
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.189@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:15.650836 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.641394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55988 (local address 127.29.84.190:44055)
0806 00:44:15.641538 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:15.641543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:15.641772 (+ 229us) server_negotiation.cc:407] Connection header received
0806 00:44:15.641887 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:15.641890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:15.641945 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:15.642029 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:15.642621 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.643221 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.644220 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.644628 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.644749 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.647269 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:15.647273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:15.647275 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:15.647305 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:15.648822 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.649268 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.649271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.649273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.649396 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.649723 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.649727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.649728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.649948 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:15.650078 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:15.650532 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 00:44:15.650679 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 00:44:15.651480 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.641309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:55988)
0806 00:44:15.641631 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:15.641677 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:15.641793 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:15.642083 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:15.642087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:15.642383 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:15.642490 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.642495 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.643362 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.643365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.644095 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.644103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.644768 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.644771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.644841 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.645291 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:15.645310 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:15.647126 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:15.648921 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.648928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.648932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.649155 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.649532 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.649536 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.649538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.649628 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.650077 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:15.650083 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:15.650212 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 00:44:15.650350 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 00:44:15.650551 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":21,"thread_start_us":109,"threads_started":1}
I20260806 00:44:15.651481 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.637628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.29.84.189:43869)
0806 00:44:15.638061 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:15.638080 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:15.638124 (+ 44us) server_negotiation.cc:407] Connection header received
0806 00:44:15.638912 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:15.638936 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:15.639355 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:15.639720 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:15.640442 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.641705 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.642695 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.643208 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.643362 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.646308 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:15.646315 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:15.646332 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:15.646352 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:15.648395 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.648814 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.648819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.648824 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.648954 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.649261 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.649266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.649268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.649704 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:15.649898 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:15.650224 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 00:44:15.650480 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":276,"thread_start_us":124,"threads_started":1}
I20260806 00:44:15.652936 7483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.189@KRBTEST.COM for oryx/127.29.84.189@KRBTEST.COM
I20260806 00:44:15.660379 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.653377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.0.0.1:35162)
0806 00:44:15.653454 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:15.653468 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:15.653535 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:15.653772 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:15.653775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:15.653925 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:15.654324 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.654330 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.655047 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.655049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.655615 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.655624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.656017 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.656019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.656088 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.656517 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:15.656529 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:15.657599 (+ 1070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:15.659045 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.659047 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.659049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.659401 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.659662 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.659664 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.659665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.659738 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.660012 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:15.660014 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:15.660081 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 00:44:15.660165 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 00:44:15.660258 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 00:44:15.660439 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.653450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35162 (local address 127.29.84.189:43869)
0806 00:44:15.653526 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:15.653536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:15.653548 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:44:15.653612 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:15.653616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:15.653667 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:15.653759 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:15.654464 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.654955 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.655715 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.655917 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.655999 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.657713 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:15.657716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:15.657718 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:15.657738 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:15.658949 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.659502 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.659505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.659506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.659579 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.659806 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.659808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.659809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.659940 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:15.660004 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:15.660195 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 00:44:15.660309 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":26}
I20260806 00:44:15.662905 7483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: Bootstrap starting.
I20260806 00:44:15.665071 7483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:15.665776 7483 log.cc:824] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:15.667549 7483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: No bootstrap required, opened a new log
I20260806 00:44:15.670315 7483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } }
I20260806 00:44:15.670517 7483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:15.670598 7483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8566684ab940cb804ba536f479df5e, State: Initialized, Role: FOLLOWER
I20260806 00:44:15.671000 7483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } }
I20260806 00:44:15.671476 7496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } } }
I20260806 00:44:15.671581 7496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:44:15.672142 7483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: configured and running, proceeding with master startup.
W20260806 00:44:15.676458 7514 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:44:15.676553 7514 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:44:15.679232 7388 master_runner.cc:428] Detected that this master 5d8566684ab940cb804ba536f479df5e is joining an existing cluster
I20260806 00:44:15.679301 7388 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.84.189:43869
I20260806 00:44:15.689905 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.681282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56002 (local address 127.29.84.190:44055)
0806 00:44:15.681421 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:15.681426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:15.681674 (+ 248us) server_negotiation.cc:407] Connection header received
0806 00:44:15.681883 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:15.681887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:15.681938 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:15.682023 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:15.682670 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.683450 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.684315 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.684546 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.684665 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.686447 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:15.686450 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:15.686453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:15.686479 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:15.688055 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.688599 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.688603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.688605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.688694 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.689008 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.689012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.689014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.689194 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:15.689309 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:15.689655 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 00:44:15.689767 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:44:15.690095 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.681672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35174 (local address 127.29.84.189:43869)
0806 00:44:15.681770 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:15.681774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:15.681788 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:44:15.681904 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:15.681908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:15.681949 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:15.682036 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:15.682592 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.683396 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.684611 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.684942 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.685044 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.686652 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:15.686657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:15.686662 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:15.686684 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:15.688413 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.688859 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.688864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.688868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.688962 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.689230 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.689233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.689235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.689435 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:15.689500 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:15.689792 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 00:44:15.689897 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":41,"spinlock_wait_cycles":2816}
I20260806 00:44:15.689925 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.681241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:56002)
0806 00:44:15.681586 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:15.681602 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:15.681745 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:15.682146 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:15.682149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:15.682397 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:15.682557 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.682565 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.683580 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.683583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.684194 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.684202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.684787 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.684790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.684923 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.685862 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:15.685881 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:15.686313 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:15.688184 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.688187 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.688189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.688443 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.688795 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.688800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.688803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.688904 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.689314 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:15.689318 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:15.689475 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 00:44:15.689601 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 00:44:15.689734 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":97,"spinlock_wait_cycles":3328,"thread_start_us":117,"threads_started":1}
I20260806 00:44:15.690268 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.681340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.0.0.1:35174)
0806 00:44:15.681706 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:15.681719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:15.681803 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:15.682042 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:15.682045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:15.682344 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:15.682479 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.682485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.683512 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.683516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.684485 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.684497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.685065 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.685069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.685492 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.686013 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:15.686029 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:15.686506 (+ 477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:15.688517 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.688521 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.688525 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.688729 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.689058 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.689060 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.689062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.689144 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.689555 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:15.689558 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:15.689643 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 00:44:15.689940 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 00:44:15.690103 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":31,"thread_start_us":95,"threads_started":1}
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977054, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.190@KRBTEST.COM for oryx/127.29.84.189@KRBTEST.COM
I20260806 00:44:15.703141 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.694575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35176 (local address 127.29.84.189:43869)
0806 00:44:15.694682 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:15.694688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:15.694976 (+ 288us) server_negotiation.cc:407] Connection header received
0806 00:44:15.695217 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:15.695220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:15.695265 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:15.695358 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:15.695942 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.696419 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.697358 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.697588 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.697693 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.700052 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:15.700055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:15.700058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:15.700087 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:15.701434 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.701889 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.701894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.701896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.701973 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.702258 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.702261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.702263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.702418 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:15.702523 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:15.702855 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 00:44:15.702957 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:44:15.703183 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.694536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.0.0.1:35176)
0806 00:44:15.694862 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:15.694900 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:15.695095 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:15.695391 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:15.695397 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:15.695733 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:15.695829 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.695834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.696523 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.696527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.697238 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.697248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.697698 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.697702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.697819 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.698345 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:15.698360 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:15.699920 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:15.701546 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.701558 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.701565 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.701789 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.702070 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.702073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.702075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.702160 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.702526 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:15.702533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:15.702660 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 00:44:15.702794 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 00:44:15.702968 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":90,"threads_started":1}
I20260806 00:44:15.704546 6640 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.84.189:43869: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5d8566684ab940cb804ba536f479df5e"
member_type: NON_VOTER
last_known_addr {
host: "127.29.84.189"
port: 43869
}
attrs {
promote: true
}
}
dest_uuid: "56329dd2abc5442fbb981a835e5df66a"
cas_config_opid_index: -1
I20260806 00:44:15.705211 6640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } }
I20260806 00:44:15.706943 7527 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5d8566684ab940cb804ba536f479df5e (127.29.84.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } } }
I20260806 00:44:15.707656 7448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:15.708978 7527 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.84.189:43869
I20260806 00:44:15.709009 7448 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Refusing update from remote peer 56329dd2abc5442fbb981a835e5df66a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 00:44:15.709744 7388 master.cc:537] Master@127.29.84.189:43869 shutting down...
I20260806 00:44:15.709746 7527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:15.710645 7525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } } }
I20260806 00:44:15.710753 7525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
W20260806 00:44:15.712811 6622 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 00:44:15.713937 6622 consensus_peers.cc:487] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a -> Peer 5d8566684ab940cb804ba536f479df5e (127.29.84.189:43869): Couldn't send request to peer 5d8566684ab940cb804ba536f479df5e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 00:44:15.714869 7388 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:15.715193 7388 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:15.715317 7388 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: stopping tablet replica
WARNING: no policy specified for oryx/127.29.84.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.188@KRBTEST.COM" created.
I20260806 00:44:15.730556 7388 master.cc:552] Master@127.29.84.189:43869 shutdown complete.
I20260806 00:44:15.730620 7388 master_runner.cc:305] Clearing existing system tablet
I20260806 00:44:15.730845 7388 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:15.730914 7388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:15.732088 7388 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:15.733019 7536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.733232 7388 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:15.733367 7388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:15.733424 7388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal
uuid: "5d8566684ab940cb804ba536f479df5e"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0f1806afaa0d74466a9737a11b099e9a"
server_key_iv: "2a85a28615c4b1a18a0a2536278df657"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.735484 7388 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.188.keytab.
Entry for principal oryx/127.29.84.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.188.keytab.
I20260806 00:44:15.737818 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.188:45167
--webserver_interface=127.29.84.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.188.keytab
--principal=oryx/127.29.84.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:44055,127.29.84.189:43869,127.29.84.188:45167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:44:15.738333 7388 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 00:44:15.738420 7388 log.cc:1196] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 00:44:15.738682 7388 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: Deleting consensus metadata
I20260806 00:44:15.738840 7388 master_runner.cc:315] Copying system tablet from 127.29.84.190:44055
I20260806 00:44:15.740165 7388 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: tablet copy: Beginning tablet copy session from remote peer at address 127.29.84.190:44055
I20260806 00:44:15.748683 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.740874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56004 (local address 127.29.84.190:44055)
0806 00:44:15.741005 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:15.741010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:15.741594 (+ 584us) server_negotiation.cc:407] Connection header received
0806 00:44:15.741786 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:15.741789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:15.741837 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:15.741937 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:15.742391 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.742920 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.743815 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.744051 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.744163 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.745365 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:15.745369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:15.745371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:15.745410 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:15.746909 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.747455 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.747459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.747461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.747577 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:15.747846 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:15.747851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:15.747853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:15.748041 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:15.748150 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:15.748408 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 00:44:15.748519 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:44:15.748698 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:15.741125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:56004)
0806 00:44:15.741480 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:15.741496 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:15.741676 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:15.741954 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:15.741957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:15.742157 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:15.742268 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.742274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.743032 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.743036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.743679 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:15.743689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:15.744166 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:15.744170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:15.744274 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:15.744845 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:15.744862 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:15.745235 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:15.747024 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.747029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.747031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.747349 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.747658 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:15.747664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:15.747667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:15.747755 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:15.748139 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:15.748142 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:15.748246 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 00:44:15.748352 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 00:44:15.748496 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":103,"threads_started":1}
I20260806 00:44:15.749575 6679 tablet_copy_service.cc:140] P 56329dd2abc5442fbb981a835e5df66a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5d8566684ab940cb804ba536f479df5e ({username='oryx', principal='oryx/127.29.84.189@KRBTEST.COM'} at 127.0.0.1:56004)
I20260806 00:44:15.749781 6679 tablet_copy_service.cc:161] P 56329dd2abc5442fbb981a835e5df66a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5d8566684ab940cb804ba536f479df5e at {username='oryx', principal='oryx/127.29.84.189@KRBTEST.COM'} at 127.0.0.1:56004: session id = 5d8566684ab940cb804ba536f479df5e-00000000000000000000000000000000
I20260806 00:44:15.751561 6679 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 00:44:15.753103 7388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 00:44:15.755259 7388 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: tablet copy: Starting download of 0 data blocks...
I20260806 00:44:15.755456 7388 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: tablet copy: Starting download of 1 WAL segments...
I20260806 00:44:15.757072 7388 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 00:44:15.758575 6679 tablet_copy_service.cc:342] P 56329dd2abc5442fbb981a835e5df66a: Request end of tablet copy session 5d8566684ab940cb804ba536f479df5e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.84.189@KRBTEST.COM'} at 127.0.0.1:56004
I20260806 00:44:15.758703 6679 tablet_copy_service.cc:433] P 56329dd2abc5442fbb981a835e5df66a: ending tablet copy session 5d8566684ab940cb804ba536f479df5e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5d8566684ab940cb804ba536f479df5e
I20260806 00:44:15.765345 7388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:15.766836 7552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:15.767155 7547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:15.767349 7388 server_base.cc:1034] running on GCE node
W20260806 00:44:15.767503 7548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:15.767758 7388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:15.768083 7388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:15.769222 7388 hybrid_clock.cc:648] HybridClock initialized: now 1785977055769214 us; error 31 us; skew 500 ppm
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:15.770687 7388 init.cc:377] Logged in from keytab as oryx/127.29.84.189@KRBTEST.COM (short username oryx)
I20260806 00:44:15.771165 7388 webserver.cc:460] Webserver started at http://127.29.84.189:34517/ using document root <none> and password file <none>
I20260806 00:44:15.771354 7388 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:15.771409 7388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:15.772490 7388 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 00:44:15.773516 7563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.773748 7388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 00:44:15.773849 7388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal
uuid: "5d8566684ab940cb804ba536f479df5e"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0f1806afaa0d74466a9737a11b099e9a"
server_key_iv: "2a85a28615c4b1a18a0a2536278df657"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.773967 7388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:15.782424 7388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:15.784381 7388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:15.789456 7388 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.189:43869
I20260806 00:44:15.790143 7388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
I20260806 00:44:15.790565 7642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.189:43869 every 8 connection(s)
I20260806 00:44:15.791849 7643 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:44:15.792635 7643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: Bootstrap starting.
I20260806 00:44:15.800477 7643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:44:15.800793 7643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: Bootstrap complete.
I20260806 00:44:15.801272 7643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } }
I20260806 00:44:15.801406 7643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5d8566684ab940cb804ba536f479df5e, State: Initialized, Role: LEARNER
I20260806 00:44:15.801563 7643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } }
I20260806 00:44:15.801888 7648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } } }
I20260806 00:44:15.802047 7648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: This master's current role is: LEARNER
I20260806 00:44:15.802047 7643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:15.804872 7669 catalog_manager.cc:1259] Loaded cluster ID: 4f5b0b9da0a54bf6bca0da60e0e9fabc
I20260806 00:44:15.804975 7669 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: loading cluster ID for follower catalog manager: success
I20260806 00:44:15.806752 7669 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: acquiring CA information for follower catalog manager: success
I20260806 00:44:15.807907 7669 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 00:44:15.906198 7540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:15.906440 7540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:15.906481 7540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:15.910441 7540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:15.910513 7540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:15.910553 7540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:15.910570 7540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:15.910633 7540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:15.915995 7540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:44055,127.29.84.189:43869,127.29.84.188:45167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.188.keytab
--principal=oryx/127.29.84.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.188:45167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.84.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:15.916989 7540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:15.918016 7540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:15.927227 7677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:15.927240 7675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:15.927268 7680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:15.927356 7540 server_base.cc:1034] running on GCE node
I20260806 00:44:15.928085 7540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:15.929081 7540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:15.930269 7540 hybrid_clock.cc:648] HybridClock initialized: now 1785977055930252 us; error 33 us; skew 500 ppm
Aug 06 00:44:15 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:15.933969 7540 init.cc:377] Logged in from keytab as oryx/127.29.84.188@KRBTEST.COM (short username oryx)
I20260806 00:44:15.934957 7540 webserver.cc:460] Webserver started at http://127.29.84.188:33919/ using document root <none> and password file <none>
I20260806 00:44:15.935518 7540 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:15.935582 7540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:15.935738 7540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:15.937907 7540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data/instance:
uuid: "70e92ce79c3941baba1206d0232d7a94"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "9527979de9f9b151840607a04f578daa"
server_key_iv: "8f962dd1ab022c10e9799e6853d3a8c7"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.938563 7540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal/instance:
uuid: "70e92ce79c3941baba1206d0232d7a94"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "9527979de9f9b151840607a04f578daa"
server_key_iv: "8f962dd1ab022c10e9799e6853d3a8c7"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.942420 7540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:44:15.944860 7690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:15.946031 7540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:44:15.946165 7540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal
uuid: "70e92ce79c3941baba1206d0232d7a94"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "9527979de9f9b151840607a04f578daa"
server_key_iv: "8f962dd1ab022c10e9799e6853d3a8c7"
server_key_version: "encryptionkey@0"
I20260806 00:44:15.946269 7540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:15.972901 7540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:15.976121 7540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:15.976362 7540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:15.990453 7540 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.188:45167
I20260806 00:44:15.990976 7766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.188:45167 every 8 connection(s)
I20260806 00:44:15.991444 7540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
I20260806 00:44:15.994403 7767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:15.998656 6934 heartbeater.cc:503] Master 127.29.84.190:44055 was elected leader, sending a full tablet report...
I20260806 00:44:15.998888 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 7540
I20260806 00:44:15.999435 7767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.84.188@KRBTEST.COM
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.188@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:16.014552 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.001785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56010 (local address 127.29.84.190:44055)
0806 00:44:16.001908 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.001912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.002562 (+ 650us) server_negotiation.cc:407] Connection header received
0806 00:44:16.003143 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.003146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.003222 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.003296 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.004781 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.005301 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.006750 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.007022 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.007191 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.010699 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.010705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.010709 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.010746 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.012472 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.013020 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.013024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.013025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.013102 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.013490 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.013492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.013494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.013660 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.013772 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.014228 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.014343 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:44:16.015407 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.001503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:56010)
0806 00:44:16.002389 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.002462 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.002782 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.003501 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.003507 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.004046 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.004617 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.004634 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.005535 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.005539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.006623 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.006632 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.007238 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.007242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.007568 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.008514 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.008532 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.010505 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.012584 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.012591 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.012598 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.012862 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.013200 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.013203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.013206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.013350 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.013769 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.013775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.014025 (+ 250us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.014187 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.014497 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"spinlock_wait_cycles":10112,"thread_start_us":269,"threads_started":1}
I20260806 00:44:16.015426 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.000464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51992 (local address 127.29.84.188:45167)
0806 00:44:16.000954 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.001001 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.001039 (+ 38us) server_negotiation.cc:407] Connection header received
0806 00:44:16.002629 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.002652 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.002918 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.003230 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.004085 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.005967 (+ 1882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.006849 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.007556 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.007669 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.010315 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.010321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.010330 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.010351 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.012680 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.013196 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.013202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.013204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.013342 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.013682 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.013687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.013689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.014009 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.014281 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.014560 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.014707 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":350,"spinlock_wait_cycles":11520,"thread_start_us":109,"threads_started":1}
I20260806 00:44:16.016801 7767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.188@KRBTEST.COM for oryx/127.29.84.189@KRBTEST.COM
I20260806 00:44:16.024988 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.017428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.0.0.1:35178)
0806 00:44:16.017568 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.017585 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.017660 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.018158 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.018160 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.018287 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.018454 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.018461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.019437 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.019440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.019993 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.020002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.020416 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.020418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.020531 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.020913 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.020923 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.022117 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.023873 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.023875 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.023876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.024051 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.024289 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.024291 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.024292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.024358 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.024659 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.024661 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.024726 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.024799 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.024890 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 00:44:16.025146 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.017617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35178 (local address 127.29.84.189:43869)
0806 00:44:16.017946 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.017949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.017961 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:44:16.017997 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.017999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.018035 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.018121 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.018569 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.019331 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.020102 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.020313 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.020397 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.022247 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.022250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.022252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.022274 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.023780 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.024142 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.024144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.024146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.024214 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.024438 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.024440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.024441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.024586 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.024655 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.024855 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.024942 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":149,"threads_started":1}
I20260806 00:44:16.025887 7767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977055, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.188@KRBTEST.COM for oryx/127.29.84.188@KRBTEST.COM
I20260806 00:44:16.033465 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.026338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.0.0.1:51996)
0806 00:44:16.026458 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.026472 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.026554 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.026774 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.026777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.026940 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.026995 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.027000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.027774 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.027780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.028266 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.028273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.028961 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.028965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.029093 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.029513 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.029533 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.030656 (+ 1123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.032086 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.032091 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.032094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.032278 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.032528 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.032532 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.032537 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.032615 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.032948 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.032950 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.033029 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.033133 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.033257 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":21}
I20260806 00:44:16.033465 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.026380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.29.84.188:45167)
0806 00:44:16.026466 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.026471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.026531 (+ 60us) server_negotiation.cc:407] Connection header received
0806 00:44:16.026647 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.026650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.026688 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.026765 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.027141 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.027589 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.028384 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.028822 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.028965 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.030782 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.030785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.030788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.030815 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.031975 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.032367 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.032373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.032374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.032441 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.032707 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.032710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.032711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.032859 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.032957 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.033186 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.033306 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":40}
I20260806 00:44:16.036819 7767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Bootstrap starting.
I20260806 00:44:16.039629 7767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:16.040699 7767 log.cc:824] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:16.042688 7767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: No bootstrap required, opened a new log
I20260806 00:44:16.046881 7767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } }
I20260806 00:44:16.047191 7767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:16.047246 7767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e92ce79c3941baba1206d0232d7a94, State: Initialized, Role: FOLLOWER
I20260806 00:44:16.047647 7767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } }
I20260806 00:44:16.048224 7774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } } }
I20260806 00:44:16.048380 7774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:44:16.048803 7767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 00:44:16.054947 7791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:44:16.055047 7791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:44:16.058463 7540 master_runner.cc:428] Detected that this master 70e92ce79c3941baba1206d0232d7a94 is joining an existing cluster
I20260806 00:44:16.058528 7540 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.84.188:45167
I20260806 00:44:16.069109 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 6739
I20260806 00:44:16.084264 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.061259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56018 (local address 127.29.84.190:44055)
0806 00:44:16.061392 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.061397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.061557 (+ 160us) server_negotiation.cc:407] Connection header received
0806 00:44:16.061729 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.061733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.061780 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.061831 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.062811 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.063785 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.065652 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.066097 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.066178 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.068451 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.068454 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.068456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.068478 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.082156 (+ 13678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.082746 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.082750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.082752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.082854 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.083213 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.083216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.083218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.083492 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.083595 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.083948 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.084073 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:44:16.084422 7799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.060979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:56018)
0806 00:44:16.061472 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.061488 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.061624 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.061977 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.061980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.062364 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.062608 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.062616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.063956 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.063959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.065442 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.065455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.066367 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.066371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.066949 (+ 578us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.067541 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.067567 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.068117 (+ 550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.082297 (+ 14180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.082303 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.082308 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.082601 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.082969 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.082975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.082980 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.083113 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.083610 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.083616 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.083739 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.083865 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.084059 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":30,"thread_start_us":863,"threads_started":1}
I20260806 00:44:16.086934 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:33915
--local_ip_for_outbound_sockets=127.29.84.129
--tserver_master_addrs=127.29.84.190:44055
--webserver_port=41849
--webserver_interface=127.29.84.129
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab
--principal=oryx/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.84.190:44055,127.29.84.189:43869
--tserver_master_addrs=127.29.84.190:44055,127.29.84.189:43869,127.29.84.188:45167 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:44:16.092782 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.061050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.0.0.1:35180)
0806 00:44:16.062274 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.062285 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.062424 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.062808 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.062811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.063299 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.063614 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.063623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.064812 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.064817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.066079 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.066090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.066800 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.066803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.067043 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.088032 (+ 20989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.088047 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.088697 (+ 650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.091207 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.091211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.091213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.091475 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.091768 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.091771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.091773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.091850 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.092282 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.092285 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.092368 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.092460 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.092589 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"mutex_wait_us":222,"spinlock_wait_cycles":4480,"thread_start_us":125,"threads_started":1}
I20260806 00:44:16.093250 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.061611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.29.84.189:43869)
0806 00:44:16.061720 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.061725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.062391 (+ 666us) server_negotiation.cc:407] Connection header received
0806 00:44:16.062512 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.062515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.062582 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.062668 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.063839 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.064679 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.066213 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.066558 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.066616 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.088870 (+ 22254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.088874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.088876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.088927 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.091089 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.091590 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.091593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.091595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.091676 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.091957 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.091960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.091962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.092157 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.092242 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.092979 (+ 737us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.093106 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:44:16.093480 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.061613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52004 (local address 127.29.84.188:45167)
0806 00:44:16.061715 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.061719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.062776 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 00:44:16.062923 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.062927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.062991 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.063062 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.064014 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.064778 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.066205 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.066586 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.066716 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.088838 (+ 22122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.088842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.088844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.088864 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.090967 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.091517 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.091521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.091525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.091600 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.091855 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.091859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.091861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.092058 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.092147 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.093129 (+ 982us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.093252 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":42}
I20260806 00:44:16.094061 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.061082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.0.0.1:52004)
0806 00:44:16.062676 (+ 1594us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.062690 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.062820 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.063256 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.063258 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.063614 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.063905 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.063914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.064907 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.064910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.066079 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.066090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.066800 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.066803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.067039 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.088247 (+ 21208us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.088262 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.088704 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.091088 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.091090 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.091093 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.091412 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.091688 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.091692 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.091694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.091772 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.092182 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.092186 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.092281 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.092367 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.092523 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1524,"mutex_wait_us":163,"spinlock_wait_cycles":1408,"thread_start_us":157,"threads_started":1}
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977054, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.190@KRBTEST.COM for oryx/127.29.84.188@KRBTEST.COM
I20260806 00:44:16.104929 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.089339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.0.0.1:52008)
0806 00:44:16.089507 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.089520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.089585 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.095795 (+ 6210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.095798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.095990 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.096092 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.096099 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.097229 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.097233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.098091 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.098101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.098790 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.098794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.098901 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.099483 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.099497 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.101195 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.103398 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.103403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.103405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.103653 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.103958 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.103961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.103962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.104056 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.104434 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.104437 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.104530 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.104626 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.104766 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 00:44:16.105561 7808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.089423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52008 (local address 127.29.84.188:45167)
0806 00:44:16.090131 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.095483 (+ 5352us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.095502 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:44:16.095570 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.095574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.095641 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.095739 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.096219 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.097091 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.098223 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.098656 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.098788 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.101367 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.101385 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.101388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.101413 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.103240 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.103752 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.103755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.103757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.103854 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.104149 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.104152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.104154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.104337 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.104427 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.105122 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.105231 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":109,"threads_started":1}
I20260806 00:44:16.106377 6640 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.84.188:45167: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "70e92ce79c3941baba1206d0232d7a94"
member_type: NON_VOTER
last_known_addr {
host: "127.29.84.188"
port: 45167
}
attrs {
promote: true
}
}
dest_uuid: "56329dd2abc5442fbb981a835e5df66a"
cas_config_opid_index: 5
I20260806 00:44:16.107014 6640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } }
I20260806 00:44:16.108805 7525 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 70e92ce79c3941baba1206d0232d7a94 (127.29.84.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.111133 7525 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.84.188:45167
I20260806 00:44:16.111526 7527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.111620 7527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:16.112020 7540 master.cc:537] Master@127.29.84.188:45167 shutting down...
W20260806 00:44:16.113755 6623 consensus_peers.cc:487] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a -> Peer 70e92ce79c3941baba1206d0232d7a94 (127.29.84.188:45167): Couldn't send request to peer 70e92ce79c3941baba1206d0232d7a94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 00:44:16.117079 7540 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:16.117360 7540 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:16.117539 7540 tablet_replica.cc:330] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: stopping tablet replica
I20260806 00:44:16.118292 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.111904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.29.84.189:43869)
0806 00:44:16.112074 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.112080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.113074 (+ 994us) server_negotiation.cc:407] Connection header received
0806 00:44:16.113234 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.113237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.113293 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.113354 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:16.114206 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.114972 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.116551 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.117239 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.117314 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.117607 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.118101 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.118176 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 00:44:16.118786 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.111608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.0.0.1:35192)
0806 00:44:16.113010 (+ 1402us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.113024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.113130 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.113985 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.113988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.114009 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:16.114087 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.114092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.115188 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.115191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.116409 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.116420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.117817 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.117821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.117919 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.117962 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.118594 (+ 632us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.118671 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1339}
I20260806 00:44:16.119414 7603 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 LEARNER]: Refusing update from remote peer 56329dd2abc5442fbb981a835e5df66a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 00:44:16.119834 7527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 00:44:16.121595 7603 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 70e92ce79c3941baba1206d0232d7a94 (127.29.84.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.123625 7648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56329dd2abc5442fbb981a835e5df66a. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.123759 7648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: This master's current role is: LEARNER
I20260806 00:44:16.123853 7527 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: attempting to promote NON_VOTER 5d8566684ab940cb804ba536f479df5e to VOTER
I20260806 00:44:16.123837 7648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: NON_VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: true } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.123898 7648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: This master's current role is: LEARNER
I20260806 00:44:16.124212 7527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } }
I20260806 00:44:16.125545 7602 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 LEARNER]: Refusing update from remote peer 56329dd2abc5442fbb981a835e5df66a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 00:44:16.125926 7812 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 00:44:16.126286 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.126023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.0.0.1:52016)
0806 00:44:16.126157 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.126219 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 00:44:16.127295 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.126781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.0.0.1:52030)
0806 00:44:16.127190 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.127239 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345}
W20260806 00:44:16.127418 6623 consensus_peers.cc:487] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a -> Peer 70e92ce79c3941baba1206d0232d7a94 (127.29.84.188:45167): Couldn't send request to peer 70e92ce79c3941baba1206d0232d7a94. Status: Network error: Client connection negotiation failed: client connection to 127.29.84.188:45167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 00:44:16.128307 7527 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5d8566684ab940cb804ba536f479df5e (127.29.84.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.128767 7602 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5d8566684ab940cb804ba536f479df5e (127.29.84.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.129721 7527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.129814 7527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:16.129918 7527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.129992 7527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:16.130378 7810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.130478 7810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:44:16.136586 7540 master.cc:552] Master@127.29.84.188:45167 shutdown complete.
I20260806 00:44:16.136639 7540 master_runner.cc:305] Clearing existing system tablet
I20260806 00:44:16.136866 7540 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:16.136920 7540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:16.138226 7540 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 00:44:16.139171 7817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.139369 7540 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:16.139496 7540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:16.139529 7540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal
uuid: "70e92ce79c3941baba1206d0232d7a94"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "9527979de9f9b151840607a04f578daa"
server_key_iv: "8f962dd1ab022c10e9799e6853d3a8c7"
server_key_version: "encryptionkey@0"
I20260806 00:44:16.141436 7540 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:16.143699 7540 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 00:44:16.143784 7540 log.cc:1196] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 00:44:16.144026 7540 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Deleting consensus metadata
I20260806 00:44:16.144191 7540 master_runner.cc:315] Copying system tablet from 127.29.84.190:44055
I20260806 00:44:16.145246 7540 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: tablet copy: Beginning tablet copy session from remote peer at address 127.29.84.190:44055
I20260806 00:44:16.154695 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.145754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:56026)
0806 00:44:16.146040 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.146055 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.146156 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.146574 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.146579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.146849 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.146984 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.146992 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.148153 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.148157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.149069 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.149081 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.149701 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.149705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.149802 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.150448 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.150462 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.150898 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.153055 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.153059 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.153061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.153311 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.153599 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.153602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.153603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.153696 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.154142 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.154145 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.154244 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.154355 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.154502 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":111,"threads_started":1}
I20260806 00:44:16.154695 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.146221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56026 (local address 127.29.84.190:44055)
0806 00:44:16.146342 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.146346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.146358 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:44:16.146393 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.146396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.146437 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.146511 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.147143 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.148021 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.149226 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.149577 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.149697 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.151039 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.151043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.151046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.151073 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.152922 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.153415 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.153418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.153420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.153507 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.153790 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.153793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.153795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.154035 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.154134 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.154429 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.154542 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:44:16.155606 6679 tablet_copy_service.cc:140] P 56329dd2abc5442fbb981a835e5df66a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 70e92ce79c3941baba1206d0232d7a94 ({username='oryx', principal='oryx/127.29.84.188@KRBTEST.COM'} at 127.0.0.1:56026)
I20260806 00:44:16.155700 6679 tablet_copy_service.cc:161] P 56329dd2abc5442fbb981a835e5df66a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 70e92ce79c3941baba1206d0232d7a94 at {username='oryx', principal='oryx/127.29.84.188@KRBTEST.COM'} at 127.0.0.1:56026: session id = 70e92ce79c3941baba1206d0232d7a94-00000000000000000000000000000000
I20260806 00:44:16.156524 6679 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 00:44:16.157663 7540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 00:44:16.159987 7540 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: tablet copy: Starting download of 0 data blocks...
I20260806 00:44:16.160176 7540 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: tablet copy: Starting download of 1 WAL segments...
I20260806 00:44:16.161538 7540 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 00:44:16.163269 6679 tablet_copy_service.cc:342] P 56329dd2abc5442fbb981a835e5df66a: Request end of tablet copy session 70e92ce79c3941baba1206d0232d7a94-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.84.188@KRBTEST.COM'} at 127.0.0.1:56026
I20260806 00:44:16.163347 6679 tablet_copy_service.cc:433] P 56329dd2abc5442fbb981a835e5df66a: ending tablet copy session 70e92ce79c3941baba1206d0232d7a94-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 70e92ce79c3941baba1206d0232d7a94
I20260806 00:44:16.170884 7540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:16.172680 7824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:16.172757 7826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:16.172994 7832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:16.173883 7540 server_base.cc:1034] running on GCE node
I20260806 00:44:16.174118 7540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:16.174459 7540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:16.175755 7540 hybrid_clock.cc:648] HybridClock initialized: now 1785977056175730 us; error 41 us; skew 500 ppm
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:16.177532 7540 init.cc:377] Logged in from keytab as oryx/127.29.84.188@KRBTEST.COM (short username oryx)
I20260806 00:44:16.178026 7540 webserver.cc:460] Webserver started at http://127.29.84.188:41881/ using document root <none> and password file <none>
I20260806 00:44:16.178201 7540 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:16.178234 7540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:16.179540 7540 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:16.184949 7843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.185297 7540 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 00:44:16.185411 7540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal
uuid: "70e92ce79c3941baba1206d0232d7a94"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "9527979de9f9b151840607a04f578daa"
server_key_iv: "8f962dd1ab022c10e9799e6853d3a8c7"
server_key_version: "encryptionkey@0"
I20260806 00:44:16.185504 7540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:16.193011 7540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:16.195410 7540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:16.201505 7540 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.188:45167
I20260806 00:44:16.202183 7540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
I20260806 00:44:16.202857 7165 heartbeater.cc:503] Master 127.29.84.190:44055 was elected leader, sending a full tablet report...
I20260806 00:44:16.205137 7931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.188:45167 every 8 connection(s)
I20260806 00:44:16.206827 7932 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:44:16.207953 7932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Bootstrap starting.
I20260806 00:44:16.220814 7932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:44:16.221150 7932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Bootstrap complete.
I20260806 00:44:16.221900 7932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } }
I20260806 00:44:16.222051 7932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70e92ce79c3941baba1206d0232d7a94, State: Initialized, Role: LEARNER
I20260806 00:44:16.222162 7932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } }
I20260806 00:44:16.222721 7932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:16.222700 7935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.223008 7935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: This master's current role is: LEARNER
I20260806 00:44:16.227814 7952 catalog_manager.cc:1259] Loaded cluster ID: 4f5b0b9da0a54bf6bca0da60e0e9fabc
I20260806 00:44:16.227905 7952 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: loading cluster ID for follower catalog manager: success
I20260806 00:44:16.230001 7952 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: acquiring CA information for follower catalog manager: success
I20260806 00:44:16.231263 7952 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 00:44:16.247853 7806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:16.248127 7806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:16.248245 7806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:16.253154 7806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:16.253229 7806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:16.253334 7806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:16.259460 7806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab
--principal=oryx/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:33915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=41849
--tserver_master_addrs=127.29.84.190:44055,127.29.84.189:43869,127.29.84.188:45167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:16.260545 7806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:16.261572 7806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:16.272584 7962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:16.272636 7957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:16.272663 7806 server_base.cc:1034] running on GCE node
W20260806 00:44:16.272593 7959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:16.273193 7806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:16.273824 7806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:16.274983 7806 hybrid_clock.cc:648] HybridClock initialized: now 1785977056274965 us; error 27 us; skew 500 ppm
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:16.278290 7806 init.cc:377] Logged in from keytab as oryx/127.29.84.129@KRBTEST.COM (short username oryx)
I20260806 00:44:16.279263 7806 webserver.cc:460] Webserver started at http://127.29.84.129:41849/ using document root <none> and password file <none>
I20260806 00:44:16.279807 7806 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:16.279930 7806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:16.283973 7806 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:16.286171 7973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.287545 7806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:44:16.287688 7806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "ca9e4be9fc804074a3d0d02d481b3b38"
format_stamp: "Formatted at 2026-08-06 00:44:14 on dist-test-slave-n326"
server_key: "66e999fe021aca821bb1918b071630fe"
server_key_iv: "57047847bee9a2d0add478664e7df966"
server_key_version: "encryptionkey@0"
I20260806 00:44:16.288228 7806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:16.302531 7806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:16.306133 7806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:16.306321 7806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:16.306998 7806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:16.307966 7806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:16.308043 7806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.308120 7806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:16.308164 7806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.325858 7806 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:33915
I20260806 00:44:16.326066 8138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:33915 every 8 connection(s)
I20260806 00:44:16.326963 7806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:16.331372 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 7806
I20260806 00:44:16.331585 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 6945
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.189@KRBTEST.COM
I20260806 00:44:16.348096 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.329444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:40499 (local address 127.29.84.189:43869)
0806 00:44:16.329586 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.329591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.330619 (+ 1028us) server_negotiation.cc:407] Connection header received
0806 00:44:16.335259 (+ 4640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.335261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.335309 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.335358 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.336253 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.337129 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.339321 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.339664 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.339752 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.343742 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.343749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.343755 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.343788 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.345485 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.346129 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.346137 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.346140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.346235 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.346660 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.346666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.346673 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.346924 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.347026 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.347755 (+ 729us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.347873 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.188@KRBTEST.COM
I20260806 00:44:16.349987 8150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.329518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.29.84.129:40499)
0806 00:44:16.330469 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.330522 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.331911 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.335531 (+ 3620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.335536 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.335809 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.336054 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.336063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.337281 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.337284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.339182 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.339195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.339904 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.339910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.340103 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.340785 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.340808 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.343524 (+ 2716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.345613 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.345621 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.345638 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.346002 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.346350 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.346356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.346360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.346534 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.347122 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.347128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.347447 (+ 319us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.348260 (+ 813us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.348589 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":636,"thread_start_us":265,"threads_started":1}
I20260806 00:44:16.350325 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.329971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:39051 (local address 127.29.84.190:44055)
0806 00:44:16.330101 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.330106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.331466 (+ 1360us) server_negotiation.cc:407] Connection header received
0806 00:44:16.332245 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.332251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.332313 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.332377 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.334398 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.335361 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.337630 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.338007 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.338083 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.345873 (+ 7790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.345876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.345880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.345912 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.348116 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.348978 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.348981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.348983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.349065 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.349362 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.349366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.349367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.349573 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.349642 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.350058 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.350179 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:44:16.350864 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.329518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.129:39051)
0806 00:44:16.331304 (+ 1786us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.331318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.332138 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.333059 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.333068 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.333656 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.334248 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.334262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.335631 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.335634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.337464 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.337482 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.338263 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.338266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.338707 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.340561 (+ 1854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.340603 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.343116 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.348570 (+ 5454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.348573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.348575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.348850 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.349174 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.349177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.349179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.349260 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.349722 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.349724 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.349819 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.350525 (+ 706us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.350663 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1691,"mutex_wait_us":224,"thread_start_us":894,"threads_started":1}
I20260806 00:44:16.351580 8141 heartbeater.cc:347] Connected to a master server at 127.29.84.189:43869
I20260806 00:44:16.351641 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:35959
--local_ip_for_outbound_sockets=127.29.84.130
--tserver_master_addrs=127.29.84.190:44055
--webserver_port=39869
--webserver_interface=127.29.84.130
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab
--principal=oryx/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.84.190:44055,127.29.84.189:43869
--tserver_master_addrs=127.29.84.190:44055,127.29.84.189:43869,127.29.84.188:45167 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:44:16.351850 8141 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:16.352221 8140 heartbeater.cc:347] Connected to a master server at 127.29.84.190:44055
I20260806 00:44:16.352303 8140 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:16.352485 8140 heartbeater.cc:511] Master 127.29.84.190:44055 requested a full tablet report, sending...
I20260806 00:44:16.352485 8141 heartbeater.cc:511] Master 127.29.84.189:43869 requested a full tablet report, sending...
I20260806 00:44:16.353883 6640 ts_manager.cc:194] Re-registered known tserver with Master: ca9e4be9fc804074a3d0d02d481b3b38 (127.29.84.129:33915)
I20260806 00:44:16.354682 6640 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:39051
I20260806 00:44:16.355197 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.329518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.29.84.129:52761)
0806 00:44:16.330880 (+ 1362us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.330902 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.332079 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.334454 (+ 2375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.334457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.334795 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.335098 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.335107 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.345697 (+ 10590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.345703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.346961 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.346975 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.347885 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.347892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.348028 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.348475 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.348494 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.350559 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.353673 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.353675 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.353676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.353858 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.354151 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.354156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.354157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.354243 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.354653 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.354656 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.354740 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.354864 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.355004 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1278,"mutex_wait_us":27,"thread_start_us":257,"threads_started":1}
I20260806 00:44:16.355254 8145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.329097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:52761 (local address 127.29.84.188:45167)
0806 00:44:16.329794 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.329798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.331020 (+ 1222us) server_negotiation.cc:407] Connection header received
0806 00:44:16.332528 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.332534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.332588 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.332643 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.343838 (+ 11195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.345568 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.347135 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.347584 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.347679 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.350708 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.350712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.350714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.350741 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.352739 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.353968 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.353971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.353973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.354059 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.354349 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.354352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.354354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.354554 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.354664 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.354946 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.355096 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":112,"threads_started":1}
I20260806 00:44:16.355823 7589 ts_manager.cc:194] Registered new tserver with Master: ca9e4be9fc804074a3d0d02d481b3b38 (127.29.84.129:33915)
I20260806 00:44:16.356734 8139 heartbeater.cc:347] Connected to a master server at 127.29.84.188:45167
I20260806 00:44:16.356842 8139 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:16.357491 8139 heartbeater.cc:511] Master 127.29.84.188:45167 requested a full tablet report, sending...
I20260806 00:44:16.358907 7861 ts_manager.cc:194] Registered new tserver with Master: ca9e4be9fc804074a3d0d02d481b3b38 (127.29.84.129:33915)
I20260806 00:44:16.418996 7377 heartbeater.cc:503] Master 127.29.84.190:44055 was elected leader, sending a full tablet report...
W20260806 00:44:16.464007 8152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:16.464268 8152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:16.464340 8152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:16.468210 8152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:16.468284 8152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:16.468369 8152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:16.473322 8152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab
--principal=oryx/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:35959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=39869
--tserver_master_addrs=127.29.84.190:44055,127.29.84.189:43869,127.29.84.188:45167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:16.474224 8152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:16.475203 8152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:16.484297 8158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:16.484297 8159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:16.484318 8161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:16.484688 8152 server_base.cc:1034] running on GCE node
I20260806 00:44:16.485075 8152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:16.485703 8152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:16.486982 8152 hybrid_clock.cc:648] HybridClock initialized: now 1785977056486948 us; error 51 us; skew 500 ppm
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:16.489857 8152 init.cc:377] Logged in from keytab as oryx/127.29.84.130@KRBTEST.COM (short username oryx)
I20260806 00:44:16.490622 8152 webserver.cc:460] Webserver started at http://127.29.84.130:39869/ using document root <none> and password file <none>
I20260806 00:44:16.491225 8152 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:16.491307 8152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:16.495285 8152 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:16.497210 8169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.498364 8152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:16.498462 8152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "be43b3654b7a495f8ab51fd621037e66"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0ca5dda42e2dae7d736195310bd715da"
server_key_iv: "d2c49c70a37a09ca624945434647610d"
server_key_version: "encryptionkey@0"
I20260806 00:44:16.498986 8152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:16.515715 8152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:16.518177 8152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:16.518314 8152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:16.518769 8152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:16.519595 8152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:16.519641 8152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.519677 8152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:16.519692 8152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.529127 8152 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:35959
I20260806 00:44:16.529155 8342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:35959 every 8 connection(s)
I20260806 00:44:16.530032 8152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:16.540154 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 8152
I20260806 00:44:16.540354 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 7172
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.189@KRBTEST.COM
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.188@KRBTEST.COM
I20260806 00:44:16.547416 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.531765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:45115 (local address 127.29.84.189:43869)
0806 00:44:16.531888 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.531892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.532959 (+ 1067us) server_negotiation.cc:407] Connection header received
0806 00:44:16.534277 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.534280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.534340 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.534425 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.535984 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.536759 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.538033 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.538267 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.538377 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.542557 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.542563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.542569 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.542598 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.544709 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.545299 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.545305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.545310 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.545415 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.545953 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.545960 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.545964 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.546263 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.546351 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.547050 (+ 699us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.547198 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:44:16.548894 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.532143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.29.84.130:45115)
0806 00:44:16.532813 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.532863 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.534058 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.534568 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.534574 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.535319 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.535834 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.535839 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.536890 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.536902 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.537908 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.537917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.538421 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.538425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.538630 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.539305 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.539326 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.542385 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.544829 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.544837 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.544851 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.545175 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.545577 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.545583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.545588 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.545819 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.546454 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.546463 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.546807 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.547616 (+ 809us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.547918 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":116,"thread_start_us":830,"threads_started":1}
I20260806 00:44:16.550004 8348 heartbeater.cc:347] Connected to a master server at 127.29.84.189:43869
I20260806 00:44:16.550196 8348 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:16.550671 8348 heartbeater.cc:511] Master 127.29.84.189:43869 requested a full tablet report, sending...
I20260806 00:44:16.551812 7589 ts_manager.cc:194] Registered new tserver with Master: be43b3654b7a495f8ab51fd621037e66 (127.29.84.130:35959)
I20260806 00:44:16.560392 8145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.532336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:34897 (local address 127.29.84.188:45167)
0806 00:44:16.532489 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.532494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.533494 (+ 1000us) server_negotiation.cc:407] Connection header received
0806 00:44:16.534305 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.534309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.534362 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.534419 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.535965 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.536475 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.538405 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.538754 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.538825 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.542618 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.542625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.542631 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.542668 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.558369 (+ 15701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.558859 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.558865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.558870 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.558963 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.559302 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.559308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.559313 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.559566 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.559664 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.560014 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.560156 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 00:44:16.561362 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.554475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.0.0.1:51478)
0806 00:44:16.554717 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.554733 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.554806 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.558290 (+ 3484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.558294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.558314 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:16.558405 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.558410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.559397 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.559401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.560130 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.560139 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.560952 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.560956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.561042 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.561064 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.561155 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.561238 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260806 00:44:16.561364 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.557800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.29.84.188:45167)
0806 00:44:16.558050 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.558053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.558065 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:44:16.558109 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.558110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.558158 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.558281 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:16.558555 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.559297 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.560255 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.560829 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.560946 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.561083 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.561173 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.561240 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":96,"threads_started":1}
I20260806 00:44:16.561434 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.532144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.29.84.130:34897)
0806 00:44:16.533377 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.533389 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.534057 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.534646 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.534650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.535262 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.535816 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.535830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.536710 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.536713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.538259 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.538271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.539011 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.539015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.539220 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.539781 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.539801 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.542435 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.558496 (+ 16061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.558500 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.558502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.558752 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.559099 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.559103 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.559105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.559198 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.559715 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.559720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.559825 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.560622 (+ 797us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.560794 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"mutex_wait_us":161,"spinlock_wait_cycles":51840,"thread_start_us":153,"threads_started":1}
I20260806 00:44:16.562103 8343 heartbeater.cc:347] Connected to a master server at 127.29.84.188:45167
I20260806 00:44:16.562215 8343 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:16.562162 7877 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 00:44:16.562402 8343 heartbeater.cc:511] Master 127.29.84.188:45167 requested a full tablet report, sending...
I20260806 00:44:16.562825 7935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56329dd2abc5442fbb981a835e5df66a. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.562989 7935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: This master's current role is: LEARNER
I20260806 00:44:16.563158 7861 ts_manager.cc:194] Registered new tserver with Master: be43b3654b7a495f8ab51fd621037e66 (127.29.84.130:35959)
I20260806 00:44:16.565428 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:33215
--local_ip_for_outbound_sockets=127.29.84.131
--tserver_master_addrs=127.29.84.190:44055
--webserver_port=46261
--webserver_interface=127.29.84.131
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab
--principal=oryx/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.84.190:44055,127.29.84.189:43869
--tserver_master_addrs=127.29.84.190:44055,127.29.84.189:43869,127.29.84.188:45167 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:16.569470 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.532144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.130:60653)
0806 00:44:16.533631 (+ 1487us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.533642 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.534059 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.534783 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.534787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.535319 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.535828 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.535838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.537067 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.537070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.538259 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.538271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.539316 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.539320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.539480 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.539956 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.539970 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.566050 (+ 26080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.568011 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.568014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.568015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.568213 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.568518 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.568520 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.568521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.568591 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.569029 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.569031 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.569109 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.569185 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.569287 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1425,"mutex_wait_us":490,"spinlock_wait_cycles":40192,"thread_start_us":130,"threads_started":1}
I20260806 00:44:16.569640 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.532325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:60653 (local address 127.29.84.190:44055)
0806 00:44:16.532448 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.532453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.533741 (+ 1288us) server_negotiation.cc:407] Connection header received
0806 00:44:16.534545 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.534550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.534620 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.534750 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.535985 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.536763 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.538404 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.538748 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.538825 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.566184 (+ 27359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.566188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.566191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.566218 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.567893 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.568310 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.568315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.568317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.568421 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.568682 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.568688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.568692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.568934 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.569041 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.569268 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.569406 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:44:16.570206 8344 heartbeater.cc:347] Connected to a master server at 127.29.84.190:44055
I20260806 00:44:16.570303 8344 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:16.570510 8344 heartbeater.cc:511] Master 127.29.84.190:44055 requested a full tablet report, sending...
I20260806 00:44:16.571254 6640 ts_manager.cc:194] Re-registered known tserver with Master: be43b3654b7a495f8ab51fd621037e66 (127.29.84.130:35959)
I20260806 00:44:16.572130 6640 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:60653
W20260806 00:44:16.677173 8356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:16.677393 8356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:16.677444 8356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:16.681447 8356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:16.681494 8356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:16.681550 8356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:16.686781 8356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab
--principal=oryx/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:33215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=46261
--tserver_master_addrs=127.29.84.190:44055,127.29.84.189:43869,127.29.84.188:45167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:16.687844 8356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:16.688771 8356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:16.698621 8362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:16.698673 8366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:16.698653 8370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:16.698823 8356 server_base.cc:1034] running on GCE node
I20260806 00:44:16.699324 8356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:16.699973 8356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:16.701161 8356 hybrid_clock.cc:648] HybridClock initialized: now 1785977056701152 us; error 28 us; skew 500 ppm
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:16.704265 8356 init.cc:377] Logged in from keytab as oryx/127.29.84.131@KRBTEST.COM (short username oryx)
I20260806 00:44:16.705077 8356 webserver.cc:460] Webserver started at http://127.29.84.131:46261/ using document root <none> and password file <none>
I20260806 00:44:16.705618 8356 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:16.705729 8356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:16.709383 8356 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:16.711401 8384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.712531 8356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:16.712687 8356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "2b9c210f183840ec93c1270003921c37"
format_stamp: "Formatted at 2026-08-06 00:44:15 on dist-test-slave-n326"
server_key: "0bfae76ef078273d8d8f157457d09b21"
server_key_iv: "bdff71b3ef859d426a693abc10d32ae5"
server_key_version: "encryptionkey@0"
I20260806 00:44:16.713239 8356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:16.733062 8356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:16.736366 8356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:16.736534 8356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:16.737171 8356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:16.737947 8356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:16.737994 8356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.738031 8356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:16.738047 8356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:16.747177 8356 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:33215
I20260806 00:44:16.747267 8554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:33215 every 8 connection(s)
I20260806 00:44:16.748202 8356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:16.752718 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 8356
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.188@KRBTEST.COM
Aug 06 00:44:16 dist-test-slave-n326 krb5kdc[6569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977056, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.189@KRBTEST.COM
I20260806 00:44:16.766867 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.750082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:47063 (local address 127.29.84.188:45167)
0806 00:44:16.750216 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.750221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.751919 (+ 1698us) server_negotiation.cc:407] Connection header received
0806 00:44:16.752488 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.752492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.752554 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.752639 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.754853 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.755600 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.757062 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.757394 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.757467 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.762018 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.762022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.762025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.762050 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.764753 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.765398 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.765402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.765404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.765480 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.765796 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.765800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.765801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.766004 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.766071 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.766573 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.766703 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:44:16.766866 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.750302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:38179 (local address 127.29.84.189:43869)
0806 00:44:16.750433 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.750437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.752753 (+ 2316us) server_negotiation.cc:407] Connection header received
0806 00:44:16.752951 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.752954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.753009 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.753099 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.754133 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.754713 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.757168 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.757491 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.757573 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.762382 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.762388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.762394 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.762422 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.764464 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.765142 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.765145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.765147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.765230 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.765630 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.765633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.765636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.765850 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.765941 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.766573 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.766696 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:44:16.768076 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.750540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.29.84.131:47063)
0806 00:44:16.751804 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.751829 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.752293 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.752890 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.752900 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.753405 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.753993 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.754018 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.755937 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.755941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.756922 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.756934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.757665 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.757671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.757901 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.759057 (+ 1156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.759095 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.761508 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.764871 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.764874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.764875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.765279 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.765601 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.765606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.765611 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.765694 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.766137 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.766139 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.766292 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.767405 (+ 1113us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.767523 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1156,"mutex_wait_us":124,"spinlock_wait_cycles":23424,"thread_start_us":194,"threads_started":1}
I20260806 00:44:16.768360 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.750787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:58369 (local address 127.29.84.190:44055)
0806 00:44:16.750952 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.750957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.751404 (+ 447us) server_negotiation.cc:407] Connection header received
0806 00:44:16.754695 (+ 3291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.754698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.754754 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.754835 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.755634 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.756467 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.757719 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.758124 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.758228 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.762164 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.762167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.762170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.762195 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.764679 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.765176 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.765180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.765182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.765261 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.765645 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.765648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.765650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.765848 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.765934 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.768070 (+ 2136us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.768185 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 00:44:16.768443 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.750754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.29.84.131:38179)
0806 00:44:16.752672 (+ 1918us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.752686 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.752820 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.753117 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.753120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.753400 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.753993 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.754015 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.755309 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.755313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.757058 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.757065 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.757712 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.757715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.757903 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.758688 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.758713 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.762202 (+ 3489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.764601 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.764606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.764620 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.765052 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.765337 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.765342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.765346 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.765524 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.766001 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.766004 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.766292 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.767055 (+ 763us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.767239 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1860,"mutex_wait_us":97,"spinlock_wait_cycles":10496,"thread_start_us":867,"threads_started":1}
I20260806 00:44:16.768378 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.750538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.131:58369)
0806 00:44:16.751259 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.751311 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.752281 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.754968 (+ 2687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.754971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.755234 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:16.755497 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.755503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.756663 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.756667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.757546 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.757557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.758371 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.758375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.758487 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.758925 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:16.758939 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:16.762020 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:16.764789 (+ 2769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.764792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.764794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.765067 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.765349 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:16.765352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:16.765354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:16.765530 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:16.765987 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:16.765993 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:16.766302 (+ 309us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.766490 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.766865 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":3,"thread_start_us":422,"threads_started":1}
I20260806 00:44:16.769783 8556 heartbeater.cc:347] Connected to a master server at 127.29.84.190:44055
I20260806 00:44:16.769799 8557 heartbeater.cc:347] Connected to a master server at 127.29.84.189:43869
I20260806 00:44:16.770042 8557 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:16.770042 8556 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:16.770443 8555 heartbeater.cc:347] Connected to a master server at 127.29.84.188:45167
I20260806 00:44:16.770560 8555 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:16.770666 8557 heartbeater.cc:511] Master 127.29.84.189:43869 requested a full tablet report, sending...
I20260806 00:44:16.770728 8555 heartbeater.cc:511] Master 127.29.84.188:45167 requested a full tablet report, sending...
I20260806 00:44:16.770684 8556 heartbeater.cc:511] Master 127.29.84.190:44055 requested a full tablet report, sending...
I20260806 00:44:16.771811 7861 ts_manager.cc:194] Registered new tserver with Master: 2b9c210f183840ec93c1270003921c37 (127.29.84.131:33215)
I20260806 00:44:16.771914 6640 ts_manager.cc:194] Re-registered known tserver with Master: 2b9c210f183840ec93c1270003921c37 (127.29.84.131:33215)
I20260806 00:44:16.772295 7589 ts_manager.cc:194] Registered new tserver with Master: 2b9c210f183840ec93c1270003921c37 (127.29.84.131:33215)
I20260806 00:44:16.772629 6640 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:58369
I20260806 00:44:16.773104 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.755780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.29.84.190:44055)
0806 00:44:16.757250 (+ 1470us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.757254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.757265 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:44:16.757298 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.757301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.757338 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.757415 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.758497 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.759324 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.760640 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.761748 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.761855 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.763918 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.763921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.763924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.763951 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.767960 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.769552 (+ 1592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.769556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.769558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.769648 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.770736 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.770739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.770741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.770943 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.772315 (+ 1372us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.772437 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.772564 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1402,"thread_start_us":65,"threads_started":1}
I20260806 00:44:16.776571 8145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.754931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.29.84.188:45167)
0806 00:44:16.755220 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.755224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.760898 (+ 5674us) server_negotiation.cc:407] Connection header received
0806 00:44:16.761122 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.761127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.761177 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.761240 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.764659 (+ 3419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.765375 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.768464 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.768792 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.768872 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.772736 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.772740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.772743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.772768 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.774693 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.775213 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.775223 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.775226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.775320 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.775629 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.775635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.775637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.775841 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.775974 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.776306 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.776419 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
I20260806 00:44:16.777808 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.755915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.29.84.189:43869)
0806 00:44:16.756676 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.756680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.758458 (+ 1778us) server_negotiation.cc:407] Connection header received
0806 00:44:16.759249 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.759252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.759305 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.759426 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.760972 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.761795 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.769240 (+ 7445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.769618 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.769718 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.773493 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.773496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.773498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.773521 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.775959 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.776455 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.776459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.776460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.776538 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.776835 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.776840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.776843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.777052 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.777176 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.777514 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.777660 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":80,"threads_started":1}
I20260806 00:44:16.776279 6640 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:44:16.780213 6640 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:16.782605 7813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: NON_VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: true } } }
I20260806 00:44:16.782728 7813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:16.785964 7813 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: attempting to promote NON_VOTER 70e92ce79c3941baba1206d0232d7a94 to VOTER
I20260806 00:44:16.786334 7813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } }
I20260806 00:44:16.787521 7877 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 LEARNER]: Refusing update from remote peer 56329dd2abc5442fbb981a835e5df66a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 00:44:16.787700 7602 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Refusing update from remote peer 56329dd2abc5442fbb981a835e5df66a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 00:44:16.788134 8579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 00:44:16.788309 7813 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 00:44:16.790148 8579 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 70e92ce79c3941baba1206d0232d7a94 (127.29.84.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:16.792059 8579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:16.792166 8579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:16.792301 8579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:16.792428 8579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:16.792554 8579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:16.792706 8579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:16.792822 7602 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 70e92ce79c3941baba1206d0232d7a94 (127.29.84.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:16.791738 7877 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 70e92ce79c3941baba1206d0232d7a94 (127.29.84.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:16.796655 8588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:16.796828 8588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:44:16.797751 7935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56329dd2abc5442fbb981a835e5df66a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:16.797880 7935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:44:16.805517 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.798172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:33915 (local address 127.0.0.1:55258)
0806 00:44:16.798328 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.798340 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.798404 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.801551 (+ 3147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.801554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.801575 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:16.801660 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.801667 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.803413 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.803417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.804289 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.804299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.805153 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.805156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.805250 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.805270 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.805332 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.805397 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":16}
I20260806 00:44:16.805730 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.798462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55258 (local address 127.29.84.129:33915)
0806 00:44:16.800962 (+ 2500us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.801013 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.801040 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:44:16.801118 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.801123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.801298 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.801812 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:16.801953 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.803281 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.804415 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.805008 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.805226 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.805368 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.805437 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.805516 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2396,"thread_start_us":69,"threads_started":1}
I20260806 00:44:16.806110 8593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.798604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:33215 (local address 127.0.0.1:45444)
0806 00:44:16.798858 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.798871 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.798961 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.800662 (+ 1701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.800665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.800683 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:16.800779 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.800784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.802371 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.802374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.803786 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.803800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.805119 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.805126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.805810 (+ 684us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.805834 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.805925 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.805988 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":1,"thread_start_us":136,"threads_started":1}
I20260806 00:44:16.806227 8594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.798757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45444 (local address 127.29.84.131:33215)
0806 00:44:16.800090 (+ 1333us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.800140 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.800166 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:44:16.800241 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.800247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.800474 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.800684 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:16.800915 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.802245 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.803997 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.804981 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.805103 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.805293 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.805958 (+ 665us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.806040 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1223,"thread_start_us":157,"threads_started":1}
I20260806 00:44:16.808915 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.799323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:35959 (local address 127.0.0.1:44884)
0806 00:44:16.800129 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.800146 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.800271 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.804719 (+ 4448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.804722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.804739 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:16.804999 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.805009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.806469 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.806472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.807512 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.807521 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.808381 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.808384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.808564 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.808586 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.808688 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.808754 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":167,"threads_started":1}
I20260806 00:44:16.809015 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.799888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.29.84.130:35959)
0806 00:44:16.804062 (+ 4174us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.804101 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.804121 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:44:16.804209 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.804214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.804374 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.804573 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:16.805133 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.806333 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.807632 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.808273 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.808380 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.808539 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.808718 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.808805 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4073,"thread_start_us":214,"threads_started":1}
I20260806 00:44:16.808916 8045 tablet_service.cc:1467] Processing CreateTablet for tablet d3b0506dd9d54b2b9a12b48e901aa407 (DEFAULT_TABLE table=test-table [id=03c5fd9e551145af85a623fefb15489c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:16.809208 8462 tablet_service.cc:1467] Processing CreateTablet for tablet d3b0506dd9d54b2b9a12b48e901aa407 (DEFAULT_TABLE table=test-table [id=03c5fd9e551145af85a623fefb15489c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:16.810153 8045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b0506dd9d54b2b9a12b48e901aa407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:16.810186 8462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b0506dd9d54b2b9a12b48e901aa407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:16.811625 8248 tablet_service.cc:1467] Processing CreateTablet for tablet d3b0506dd9d54b2b9a12b48e901aa407 (DEFAULT_TABLE table=test-table [id=03c5fd9e551145af85a623fefb15489c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:16.812599 8248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b0506dd9d54b2b9a12b48e901aa407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:16.820431 8601 tablet_bootstrap.cc:492] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: Bootstrap starting.
I20260806 00:44:16.821587 8602 tablet_bootstrap.cc:492] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: Bootstrap starting.
I20260806 00:44:16.821587 8603 tablet_bootstrap.cc:492] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: Bootstrap starting.
I20260806 00:44:16.822723 8601 tablet_bootstrap.cc:654] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:16.823930 8601 log.cc:824] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:16.824806 8602 tablet_bootstrap.cc:654] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:16.824806 8603 tablet_bootstrap.cc:654] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:16.826112 8601 tablet_bootstrap.cc:492] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: No bootstrap required, opened a new log
I20260806 00:44:16.826318 8601 ts_tablet_manager.cc:1397] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 00:44:16.826462 8602 log.cc:824] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:16.826485 8603 log.cc:824] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:16.828743 8602 tablet_bootstrap.cc:492] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: No bootstrap required, opened a new log
I20260806 00:44:16.828743 8603 tablet_bootstrap.cc:492] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: No bootstrap required, opened a new log
I20260806 00:44:16.828920 8602 ts_tablet_manager.cc:1397] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260806 00:44:16.828933 8603 ts_tablet_manager.cc:1397] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 00:44:16.832589 8601 raft_consensus.cc:348] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:16.832831 8601 raft_consensus.cc:374] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:16.832885 8601 raft_consensus.cc:729] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be43b3654b7a495f8ab51fd621037e66, State: Initialized, Role: FOLLOWER
I20260806 00:44:16.833360 8601 consensus_queue.cc:260] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:16.834419 8601 ts_tablet_manager.cc:1428] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260806 00:44:16.834652 8344 heartbeater.cc:503] Master 127.29.84.190:44055 was elected leader, sending a full tablet report...
I20260806 00:44:16.837121 8602 raft_consensus.cc:348] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:16.837129 8603 raft_consensus.cc:348] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:16.837370 8603 raft_consensus.cc:374] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:16.837370 8602 raft_consensus.cc:374] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:16.837497 8602 raft_consensus.cc:729] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9c210f183840ec93c1270003921c37, State: Initialized, Role: FOLLOWER
I20260806 00:44:16.837428 8603 raft_consensus.cc:729] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca9e4be9fc804074a3d0d02d481b3b38, State: Initialized, Role: FOLLOWER
I20260806 00:44:16.837982 8602 consensus_queue.cc:260] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:16.838006 8603 consensus_queue.cc:260] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:16.839108 8603 ts_tablet_manager.cc:1428] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260806 00:44:16.839108 8602 ts_tablet_manager.cc:1428] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: Time spent starting tablet: real 0.010s user 0.001s sys 0.010s
I20260806 00:44:16.839434 8556 heartbeater.cc:503] Master 127.29.84.190:44055 was elected leader, sending a full tablet report...
I20260806 00:44:16.841449 8140 heartbeater.cc:503] Master 127.29.84.190:44055 was elected leader, sending a full tablet report...
I20260806 00:44:16.885581 8612 raft_consensus.cc:482] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:16.885816 8612 raft_consensus.cc:504] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:16.886981 8612 leader_election.cc:288] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca9e4be9fc804074a3d0d02d481b3b38 (127.29.84.129:33915), 2b9c210f183840ec93c1270003921c37 (127.29.84.131:33215)
I20260806 00:44:16.890956 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.887352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:36309 (local address 127.29.84.129:33915)
0806 00:44:16.887475 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.887478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.887488 (+ 10us) server_negotiation.cc:407] Connection header received
0806 00:44:16.887522 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.887524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.887555 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.887635 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:16.888041 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.888540 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.889793 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.890231 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.890358 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.890409 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.890753 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.890830 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:44:16.891045 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.887168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:33915 (local address 127.29.84.130:36309)
0806 00:44:16.887264 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.887280 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.887360 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.887646 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.887649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.887665 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:16.887919 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.887927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.888653 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.888659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.889666 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.889675 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.890358 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.890361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.890546 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.890588 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.890672 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.890758 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":43}
I20260806 00:44:16.891100 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.887242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:33215 (local address 127.29.84.130:42375)
0806 00:44:16.887398 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:16.887413 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:16.887486 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:16.887732 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:16.887735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:16.887750 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:16.887937 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.887944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.888734 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.888738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.889715 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:16.889724 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.890578 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.890581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:16.890748 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.890769 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:44:16.890838 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 00:44:16.890891 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":34}
I20260806 00:44:16.891469 8594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.887315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:42375 (local address 127.29.84.131:33215)
0806 00:44:16.887469 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.887473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.887486 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:44:16.887599 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.887602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.887645 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.887722 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:16.888056 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.888615 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.889832 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.890428 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.890581 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.890650 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.891249 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.891345 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:44:16.892092 8488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b0506dd9d54b2b9a12b48e901aa407" candidate_uuid: "be43b3654b7a495f8ab51fd621037e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9c210f183840ec93c1270003921c37" is_pre_election: true
I20260806 00:44:16.892103 8070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b0506dd9d54b2b9a12b48e901aa407" candidate_uuid: "be43b3654b7a495f8ab51fd621037e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" is_pre_election: true
I20260806 00:44:16.892382 8488 raft_consensus.cc:2454] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be43b3654b7a495f8ab51fd621037e66 in term 0.
I20260806 00:44:16.892421 8070 raft_consensus.cc:2454] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be43b3654b7a495f8ab51fd621037e66 in term 0.
I20260806 00:44:16.892944 8174 leader_election.cc:302] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9c210f183840ec93c1270003921c37, be43b3654b7a495f8ab51fd621037e66; no voters:
I20260806 00:44:16.893236 8612 raft_consensus.cc:2781] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:16.893332 8612 raft_consensus.cc:482] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:16.893378 8612 raft_consensus.cc:3037] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:16.895197 8612 raft_consensus.cc:504] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:16.895572 8612 leader_election.cc:288] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [CANDIDATE]: Term 1 election: Requested vote from peers ca9e4be9fc804074a3d0d02d481b3b38 (127.29.84.129:33915), 2b9c210f183840ec93c1270003921c37 (127.29.84.131:33215)
I20260806 00:44:16.895850 8070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b0506dd9d54b2b9a12b48e901aa407" candidate_uuid: "be43b3654b7a495f8ab51fd621037e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca9e4be9fc804074a3d0d02d481b3b38"
I20260806 00:44:16.895968 8488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b0506dd9d54b2b9a12b48e901aa407" candidate_uuid: "be43b3654b7a495f8ab51fd621037e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9c210f183840ec93c1270003921c37"
I20260806 00:44:16.895992 8070 raft_consensus.cc:3037] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:16.896091 8488 raft_consensus.cc:3037] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:16.897910 8070 raft_consensus.cc:2454] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be43b3654b7a495f8ab51fd621037e66 in term 1.
I20260806 00:44:16.898198 8488 raft_consensus.cc:2454] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be43b3654b7a495f8ab51fd621037e66 in term 1.
I20260806 00:44:16.898196 8178 leader_election.cc:302] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be43b3654b7a495f8ab51fd621037e66, ca9e4be9fc804074a3d0d02d481b3b38; no voters:
I20260806 00:44:16.898413 8612 raft_consensus.cc:2781] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:16.898667 8612 raft_consensus.cc:686] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 1 LEADER]: Becoming Leader. State: Replica: be43b3654b7a495f8ab51fd621037e66, State: Running, Role: LEADER
I20260806 00:44:16.899041 8612 consensus_queue.cc:237] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:16.902556 6640 catalog_manager.cc:5495] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 reported cstate change: term changed from 0 to 1, leader changed from <none> to be43b3654b7a495f8ab51fd621037e66 (127.29.84.130). New cstate: current_term: 1 leader_uuid: "be43b3654b7a495f8ab51fd621037e66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } health_report { overall_health: HEALTHY } } }
I20260806 00:44:16.922441 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.918687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.29.84.130:35959)
0806 00:44:16.918825 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.918829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.918843 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:44:16.919043 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.919045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.919147 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.919247 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:16.919598 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.920076 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.920939 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.921171 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.921279 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.921684 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:16.922044 (+ 360us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:16.922176 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.922247 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.922298 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:44:16.928699 8070 raft_consensus.cc:1261] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [term 1 FOLLOWER]: Refusing update from remote peer be43b3654b7a495f8ab51fd621037e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:16.928699 8488 raft_consensus.cc:1261] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [term 1 FOLLOWER]: Refusing update from remote peer be43b3654b7a495f8ab51fd621037e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:16.929433 8612 consensus_queue.cc:1035] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:16.929643 8615 consensus_queue.cc:1035] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:16.937628 8620 mvcc.cc:205] Tried to move back new op lower bound from 7315362025172484096 to 7315362025061158912. Current Snapshot: MvccSnapshot[applied={T|T < 7315362025172484096}]
I20260806 00:44:16.938174 8621 mvcc.cc:205] Tried to move back new op lower bound from 7315362025172484096 to 7315362025061158912. Current Snapshot: MvccSnapshot[applied={T|T < 7315362025172484096}]
I20260806 00:44:16.939913 8622 mvcc.cc:205] Tried to move back new op lower bound from 7315362025172484096 to 7315362025061158912. Current Snapshot: MvccSnapshot[applied={T|T < 7315362025172484096}]
I20260806 00:44:16.948990 6633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:16.949118 6633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:16.953441 6633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56329dd2abc5442fbb981a835e5df66a: Sending DeleteTablet for 3 replicas of tablet d3b0506dd9d54b2b9a12b48e901aa407
I20260806 00:44:16.954440 8045 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b0506dd9d54b2b9a12b48e901aa407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:16 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:55258
I20260806 00:44:16.954599 8462 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b0506dd9d54b2b9a12b48e901aa407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:16 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:45444
I20260806 00:44:16.955159 8634 tablet_replica.cc:330] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: stopping tablet replica
I20260806 00:44:16.955291 8248 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b0506dd9d54b2b9a12b48e901aa407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:16 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:44884
I20260806 00:44:16.955538 8634 raft_consensus.cc:2229] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:16.955838 8634 raft_consensus.cc:2258] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:16.956471 8635 tablet_replica.cc:330] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: stopping tablet replica
I20260806 00:44:16.956768 8635 raft_consensus.cc:2229] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:16.957124 8635 raft_consensus.cc:2258] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:16.957559 8634 ts_tablet_manager.cc:1905] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:16.958732 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 6594
I20260806 00:44:16.963424 8635 ts_tablet_manager.cc:1905] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:16.967438 8635 ts_tablet_manager.cc:1918] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:16.967556 8635 log.cc:1196] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/d3b0506dd9d54b2b9a12b48e901aa407
I20260806 00:44:16.967971 8635 ts_tablet_manager.cc:1939] T d3b0506dd9d54b2b9a12b48e901aa407 P 2b9c210f183840ec93c1270003921c37: Deleting consensus metadata
I20260806 00:44:16.968197 8634 ts_tablet_manager.cc:1918] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:16.968312 8634 log.cc:1196] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/d3b0506dd9d54b2b9a12b48e901aa407
I20260806 00:44:16.968668 8634 ts_tablet_manager.cc:1939] T d3b0506dd9d54b2b9a12b48e901aa407 P ca9e4be9fc804074a3d0d02d481b3b38: Deleting consensus metadata
I20260806 00:44:16.969781 8636 tablet_replica.cc:330] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: stopping tablet replica
I20260806 00:44:16.970105 8636 raft_consensus.cc:2229] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:44:16.970450 8636 raft_consensus.cc:2258] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:16.972046 8636 ts_tablet_manager.cc:1905] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 00:44:16.975293 8389 connection.cc:535] server connection from 127.0.0.1:45444 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 00:44:16.975680 7974 connection.cc:535] server connection from 127.0.0.1:55258 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 00:44:16.976063 7565 connection.cc:535] server connection from 127.0.0.1:35192 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:44:16.976339 8636 ts_tablet_manager.cc:1918] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:16.976496 8636 log.cc:1196] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/d3b0506dd9d54b2b9a12b48e901aa407
I20260806 00:44:16.976909 8636 ts_tablet_manager.cc:1939] T d3b0506dd9d54b2b9a12b48e901aa407 P be43b3654b7a495f8ab51fd621037e66: Deleting consensus metadata
W20260806 00:44:16.978267 8174 connection.cc:441] server connection from 127.0.0.1:44884 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44884 (request call id 1) could send its response
I20260806 00:44:16.981143 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.966612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.29.84.189:43869)
0806 00:44:16.966790 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.966794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.970912 (+ 4118us) server_negotiation.cc:407] Connection header received
0806 00:44:16.971137 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.971140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.971190 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.971242 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.972454 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.973424 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.974679 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.975044 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.975139 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.977276 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.977280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.977284 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.977309 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.979051 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.979644 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.979648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.979650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.979745 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.980096 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.980100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.980101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.980334 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.980456 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.980839 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.980982 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:44:16.990711 8145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.959184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51498 (local address 127.29.84.188:45167)
0806 00:44:16.959338 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.959343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.982301 (+ 22958us) server_negotiation.cc:407] Connection header received
0806 00:44:16.982469 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.982472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.982516 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.982573 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.983383 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.984055 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.985130 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.985433 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.985546 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.987057 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.987084 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.987087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.987106 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.988781 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.989248 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.989251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.989253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.989335 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.989666 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.989669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.989671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.989857 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.989960 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.990439 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.990553 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 00:44:16.991560 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.980180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38860 (local address 127.29.84.189:43869)
0806 00:44:16.980286 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.980290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.980465 (+ 175us) server_negotiation.cc:407] Connection header received
0806 00:44:16.980632 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.980634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.980675 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.980720 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.981993 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.982768 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.984560 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.985182 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.985256 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.987188 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.987192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.987194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.987219 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.989033 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.989574 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.989577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.989579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.989663 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.990140 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.990144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.990148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.990449 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.990548 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.991233 (+ 685us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.991362 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 00:44:16.991886 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:16.979781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51512 (local address 127.29.84.188:45167)
0806 00:44:16.979932 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:16.979936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:16.980853 (+ 917us) server_negotiation.cc:407] Connection header received
0806 00:44:16.981042 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:16.981045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:16.981090 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:16.981137 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:16.982061 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.982797 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.984011 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:16.984319 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:16.984444 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:16.986955 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:16.986961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:16.986967 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:16.986998 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:16.989567 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.990213 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.990218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.990220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.990317 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:16.990661 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:16.990665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:16.990666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:16.990885 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:16.991001 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:16.991619 (+ 618us) server_negotiation.cc:299] Negotiation successful
0806 00:44:16.991742 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:44:17.940758 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:17.940320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.129:34129)
0806 00:44:17.940586 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:17.940666 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":100,"threads_started":1}
W20260806 00:44:17.941109 8140 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:44055 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
I20260806 00:44:17.942338 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:17.941843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.130:55173)
0806 00:44:17.942144 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:17.942238 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":68,"threads_started":1}
W20260806 00:44:17.942655 8344 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:44055 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
I20260806 00:44:17.943612 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:17.943151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.131:37341)
0806 00:44:17.943456 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:17.943537 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":66,"threads_started":1}
W20260806 00:44:17.943862 8556 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:44055 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
I20260806 00:44:18.454814 8651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56329dd2abc5442fbb981a835e5df66a)
I20260806 00:44:18.455055 8651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } }
I20260806 00:44:18.456066 8651 leader_election.cc:288] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56329dd2abc5442fbb981a835e5df66a (127.29.84.190:44055), 5d8566684ab940cb804ba536f479df5e (127.29.84.189:43869)
I20260806 00:44:18.456207 8658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56329dd2abc5442fbb981a835e5df66a)
I20260806 00:44:18.456418 8658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } }
I20260806 00:44:18.456751 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.456138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:38864)
0806 00:44:18.456576 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.456655 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":158,"threads_started":1}
W20260806 00:44:18.456919 7844 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
I20260806 00:44:18.457362 8658 leader_election.cc:288] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56329dd2abc5442fbb981a835e5df66a (127.29.84.190:44055), 70e92ce79c3941baba1206d0232d7a94 (127.29.84.188:45167)
I20260806 00:44:18.457974 8667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.457435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:38870)
0806 00:44:18.457802 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.457864 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":71,"threads_started":1}
I20260806 00:44:18.458148 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.457583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:38876)
0806 00:44:18.458005 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.458057 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329}
W20260806 00:44:18.458190 7564 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
W20260806 00:44:18.458256 7844 leader_election.cc:334] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 56329dd2abc5442fbb981a835e5df66a (127.29.84.190:44055): Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
I20260806 00:44:18.458985 8667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.458769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:38888)
0806 00:44:18.458875 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.458924 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:44:18.459112 7564 leader_election.cc:334] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 56329dd2abc5442fbb981a835e5df66a (127.29.84.190:44055): Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
I20260806 00:44:18.461266 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.456290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43869 (local address 127.0.0.1:38870)
0806 00:44:18.456712 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.456733 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:18.456945 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:18.457337 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:18.457341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:18.457362 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:18.457493 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.457503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.458892 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.458896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.460245 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.460256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.460969 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.460972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.461034 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.461054 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:44:18.461110 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 00:44:18.461161 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":61,"threads_started":1}
I20260806 00:44:18.461356 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.456368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.29.84.189:43869)
0806 00:44:18.456671 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:18.456676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:18.456807 (+ 131us) server_negotiation.cc:407] Connection header received
0806 00:44:18.457111 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:18.457114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:18.457188 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:18.457273 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:18.457737 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.458516 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.460360 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.460852 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.460954 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.461010 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:18.461145 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 00:44:18.461234 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":230,"thread_start_us":52,"threads_started":1}
I20260806 00:44:18.461763 7602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70e92ce79c3941baba1206d0232d7a94" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5d8566684ab940cb804ba536f479df5e" is_pre_election: true
I20260806 00:44:18.461968 7602 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70e92ce79c3941baba1206d0232d7a94 in term 1.
I20260806 00:44:18.462107 8668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.457675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:45167 (local address 127.0.0.1:51526)
0806 00:44:18.457938 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.457970 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:18.458050 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:18.458471 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:18.458475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:18.458494 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:18.458583 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.458590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.459516 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.459520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.460660 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.460669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.461677 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.461680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.461763 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.461793 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 00:44:18.461873 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 00:44:18.461947 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":27,"thread_start_us":60,"threads_started":1}
I20260806 00:44:18.462174 8669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.457798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.29.84.188:45167)
0806 00:44:18.458232 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:18.458242 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:18.458255 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:44:18.458296 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:18.458298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:18.458339 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:18.458714 (+ 375us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:18.458792 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.459409 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.460776 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.461545 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.461646 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.461702 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:18.461934 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 00:44:18.462004 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":85,"threads_started":1}
I20260806 00:44:18.462425 7847 leader_election.cc:302] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d8566684ab940cb804ba536f479df5e, 70e92ce79c3941baba1206d0232d7a94; no voters: 56329dd2abc5442fbb981a835e5df66a
I20260806 00:44:18.462555 7877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d8566684ab940cb804ba536f479df5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "70e92ce79c3941baba1206d0232d7a94" is_pre_election: true
I20260806 00:44:18.462651 8651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 00:44:18.462721 8651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56329dd2abc5442fbb981a835e5df66a)
I20260806 00:44:18.462769 8651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:44:18.463876 8651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } }
I20260806 00:44:18.464110 7877 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5d8566684ab940cb804ba536f479df5e in current term 2: Already voted for candidate 70e92ce79c3941baba1206d0232d7a94 in this term.
I20260806 00:44:18.464280 8651 leader_election.cc:288] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [CANDIDATE]: Term 2 election: Requested vote from peers 56329dd2abc5442fbb981a835e5df66a (127.29.84.190:44055), 5d8566684ab940cb804ba536f479df5e (127.29.84.189:43869)
I20260806 00:44:18.464639 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.464363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:38904)
0806 00:44:18.464528 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.464579 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 00:44:18.464618 7602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70e92ce79c3941baba1206d0232d7a94" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5d8566684ab940cb804ba536f479df5e"
I20260806 00:44:18.464731 7602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:44:18.464828 7568 leader_election.cc:302] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d8566684ab940cb804ba536f479df5e; no voters: 56329dd2abc5442fbb981a835e5df66a, 70e92ce79c3941baba1206d0232d7a94
I20260806 00:44:18.465312 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.465103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:38906)
0806 00:44:18.465209 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.465259 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 00:44:18.465430 7844 leader_election.cc:334] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 56329dd2abc5442fbb981a835e5df66a (127.29.84.190:44055): Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
I20260806 00:44:18.465840 7602 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70e92ce79c3941baba1206d0232d7a94 in term 2.
I20260806 00:44:18.465951 8658 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:44:18.466161 7847 leader_election.cc:302] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d8566684ab940cb804ba536f479df5e, 70e92ce79c3941baba1206d0232d7a94; no voters: 56329dd2abc5442fbb981a835e5df66a
I20260806 00:44:18.466324 8651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:44:18.466523 8651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [term 2 LEADER]: Becoming Leader. State: Replica: 70e92ce79c3941baba1206d0232d7a94, State: Running, Role: LEADER
I20260806 00:44:18.466781 8651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } }
I20260806 00:44:18.467852 8671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70e92ce79c3941baba1206d0232d7a94. Latest consensus state: current_term: 2 leader_uuid: "70e92ce79c3941baba1206d0232d7a94" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:18.467975 8671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:18.468250 8676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:18.470347 8676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:18.470757 8676 catalog_manager.cc:1259] Loaded cluster ID: 4f5b0b9da0a54bf6bca0da60e0e9fabc
I20260806 00:44:18.470803 8676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:18.471382 8676 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:18.471777 8676 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Loaded TSK: 0
I20260806 00:44:18.472407 8676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:44:18.506760 7861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:44:18.509819 7861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:18.512645 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.512356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:38914)
0806 00:44:18.512488 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.512561 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 00:44:18.512732 7602 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [term 2 FOLLOWER]: Refusing update from remote peer 70e92ce79c3941baba1206d0232d7a94: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 00:44:18.513348 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.513120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.0.0.1:38930)
0806 00:44:18.513209 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.513261 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 00:44:18.513271 8671 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 00:44:18.513510 7844 consensus_peers.cc:487] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 -> Peer 56329dd2abc5442fbb981a835e5df66a (127.29.84.190:44055): Couldn't send request to peer 56329dd2abc5442fbb981a835e5df66a. Status: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 00:44:18.514745 8658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70e92ce79c3941baba1206d0232d7a94. Latest consensus state: current_term: 2 leader_uuid: "70e92ce79c3941baba1206d0232d7a94" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:18.514873 8658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:44:18.516045 8671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "70e92ce79c3941baba1206d0232d7a94" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:18.516193 8671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:18.516623 8658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "70e92ce79c3941baba1206d0232d7a94" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "56329dd2abc5442fbb981a835e5df66a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44055 } } peers { permanent_uuid: "5d8566684ab940cb804ba536f479df5e" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43869 } attrs { promote: false } } peers { permanent_uuid: "70e92ce79c3941baba1206d0232d7a94" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 45167 } attrs { promote: false } } }
I20260806 00:44:18.516741 8658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8566684ab940cb804ba536f479df5e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:44:18.516917 8678 mvcc.cc:205] Tried to move back new op lower bound from 7315362031663403008 to 7315362031482511360. Current Snapshot: MvccSnapshot[applied={T|T < 7315362031663403008}]
I20260806 00:44:18.526187 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.521791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:33915 (local address 127.0.0.1:55268)
0806 00:44:18.522223 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.522238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:18.522328 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:18.522802 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:18.522824 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:18.522842 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:18.523001 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.523008 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.524116 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.524121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.525058 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.525067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.525722 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.525726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.525829 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.525854 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:44:18.525939 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 00:44:18.526050 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":130}
I20260806 00:44:18.526640 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.521894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55268 (local address 127.29.84.129:33915)
0806 00:44:18.522531 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:18.522538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:18.522567 (+ 29us) server_negotiation.cc:407] Connection header received
0806 00:44:18.522618 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:18.522622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:18.522673 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:18.523314 (+ 641us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:18.523444 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.523945 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.525175 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.525620 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.525736 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.525809 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:18.526059 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 00:44:18.526146 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":102,"threads_started":1}
I20260806 00:44:18.526932 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.522108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:33215 (local address 127.0.0.1:45456)
0806 00:44:18.522289 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.522306 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:18.522385 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:18.522808 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:18.522811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:18.522832 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:18.523142 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.523148 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.524102 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.524105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.525390 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.525400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.526488 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.526492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.526576 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.526599 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:44:18.526668 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 00:44:18.526769 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":226,"spinlock_wait_cycles":2944}
I20260806 00:44:18.527259 8045 tablet_service.cc:1467] Processing CreateTablet for tablet 5e995c0ebb5e4ef790067ffe1a65f33f (DEFAULT_TABLE table=test-table [id=6d2346a6f984480e922c77d30f9b9f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:18.527359 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.522213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.29.84.131:33215)
0806 00:44:18.522493 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:18.522497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:18.522508 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:44:18.522553 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:18.522587 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:18.522646 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:18.522957 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:18.523257 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.523990 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.525665 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.526277 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.526380 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.526435 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:18.527119 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 00:44:18.527221 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":101,"threads_started":1}
I20260806 00:44:18.527581 8045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e995c0ebb5e4ef790067ffe1a65f33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:18.528482 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.522627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:35959 (local address 127.0.0.1:44896)
0806 00:44:18.523031 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.523045 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:18.523152 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:18.524462 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:18.524465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:18.524482 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:18.524589 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.524605 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.525892 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.525895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.527310 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:18.527320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.528201 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.528204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:18.528268 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.528286 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:44:18.528348 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 00:44:18.528395 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260806 00:44:18.528579 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.522757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44896 (local address 127.29.84.130:35959)
0806 00:44:18.524142 (+ 1385us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:18.524148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:18.524162 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:44:18.524212 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:18.524215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:18.524266 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:18.524354 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:18.524733 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.525532 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.527486 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.528080 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.528188 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.528243 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:18.528384 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 00:44:18.528460 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":85,"threads_started":1}
I20260806 00:44:18.529095 8248 tablet_service.cc:1467] Processing CreateTablet for tablet 5e995c0ebb5e4ef790067ffe1a65f33f (DEFAULT_TABLE table=test-table [id=6d2346a6f984480e922c77d30f9b9f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:18.529347 8248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e995c0ebb5e4ef790067ffe1a65f33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:18.530053 8690 tablet_bootstrap.cc:492] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38: Bootstrap starting.
I20260806 00:44:18.530400 8462 tablet_service.cc:1467] Processing CreateTablet for tablet 5e995c0ebb5e4ef790067ffe1a65f33f (DEFAULT_TABLE table=test-table [id=6d2346a6f984480e922c77d30f9b9f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:18.530642 8462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e995c0ebb5e4ef790067ffe1a65f33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:18.530930 8690 tablet_bootstrap.cc:654] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:18.531795 8692 tablet_bootstrap.cc:492] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: Bootstrap starting.
I20260806 00:44:18.532660 8692 tablet_bootstrap.cc:654] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:18.533202 8693 tablet_bootstrap.cc:492] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: Bootstrap starting.
I20260806 00:44:18.534087 8693 tablet_bootstrap.cc:654] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:18.534376 8690 tablet_bootstrap.cc:492] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38: No bootstrap required, opened a new log
I20260806 00:44:18.534482 8690 ts_tablet_manager.cc:1397] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 00:44:18.534883 8690 raft_consensus.cc:348] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:18.534983 8690 raft_consensus.cc:374] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:18.535036 8690 raft_consensus.cc:729] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca9e4be9fc804074a3d0d02d481b3b38, State: Initialized, Role: FOLLOWER
I20260806 00:44:18.535183 8690 consensus_queue.cc:260] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:18.535373 8693 tablet_bootstrap.cc:492] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: No bootstrap required, opened a new log
I20260806 00:44:18.535472 8693 ts_tablet_manager.cc:1397] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:18.535537 8692 tablet_bootstrap.cc:492] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: No bootstrap required, opened a new log
I20260806 00:44:18.535563 8690 ts_tablet_manager.cc:1428] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:44:18.535620 8692 ts_tablet_manager.cc:1397] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 00:44:18.535997 8692 raft_consensus.cc:348] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:18.536120 8692 raft_consensus.cc:374] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:18.536163 8692 raft_consensus.cc:729] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be43b3654b7a495f8ab51fd621037e66, State: Initialized, Role: FOLLOWER
I20260806 00:44:18.536291 8692 consensus_queue.cc:260] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:18.536609 8692 ts_tablet_manager.cc:1428] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:18.537252 8693 raft_consensus.cc:348] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:18.537352 8693 raft_consensus.cc:374] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:18.537384 8693 raft_consensus.cc:729] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9c210f183840ec93c1270003921c37, State: Initialized, Role: FOLLOWER
I20260806 00:44:18.537494 8693 consensus_queue.cc:260] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:18.537823 8693 ts_tablet_manager.cc:1428] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:18.537976 8139 heartbeater.cc:503] Master 127.29.84.188:45167 was elected leader, sending a full tablet report...
I20260806 00:44:18.538560 8703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.536968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.130:55757)
0806 00:44:18.538417 (+ 1449us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.538474 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1390,"thread_start_us":90,"threads_started":1}
I20260806 00:44:18.538602 8343 heartbeater.cc:503] Master 127.29.84.188:45167 was elected leader, sending a full tablet report...
I20260806 00:44:18.539361 8703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.539143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.130:50855)
0806 00:44:18.539251 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.539300 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 00:44:18.539808 8555 heartbeater.cc:503] Master 127.29.84.188:45167 was elected leader, sending a full tablet report...
I20260806 00:44:18.540491 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.535981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.129:41385)
0806 00:44:18.540334 (+ 4353us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.540390 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4296,"thread_start_us":88,"threads_started":1}
I20260806 00:44:18.541575 8708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.541132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.129:48939)
0806 00:44:18.541429 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.541489 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":93,"threads_started":1}
I20260806 00:44:18.541656 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.540324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.131:47873)
0806 00:44:18.541489 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.541545 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1073,"thread_start_us":93,"threads_started":1}
I20260806 00:44:18.542210 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.542027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.131:47273)
0806 00:44:18.542122 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.542165 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 00:44:18.548298 8702 raft_consensus.cc:482] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:18.548408 8702 raft_consensus.cc:504] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:18.548774 8702 leader_election.cc:288] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca9e4be9fc804074a3d0d02d481b3b38 (127.29.84.129:33915), 2b9c210f183840ec93c1270003921c37 (127.29.84.131:33215)
I20260806 00:44:18.549079 8488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e995c0ebb5e4ef790067ffe1a65f33f" candidate_uuid: "be43b3654b7a495f8ab51fd621037e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9c210f183840ec93c1270003921c37" is_pre_election: true
I20260806 00:44:18.549079 8070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e995c0ebb5e4ef790067ffe1a65f33f" candidate_uuid: "be43b3654b7a495f8ab51fd621037e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" is_pre_election: true
I20260806 00:44:18.549204 8070 raft_consensus.cc:2454] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be43b3654b7a495f8ab51fd621037e66 in term 0.
I20260806 00:44:18.549204 8488 raft_consensus.cc:2454] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be43b3654b7a495f8ab51fd621037e66 in term 0.
I20260806 00:44:18.549470 8174 leader_election.cc:302] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9c210f183840ec93c1270003921c37, be43b3654b7a495f8ab51fd621037e66; no voters:
I20260806 00:44:18.549664 8702 raft_consensus.cc:2781] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:18.549721 8702 raft_consensus.cc:482] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:18.549749 8702 raft_consensus.cc:3037] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:18.550618 8702 raft_consensus.cc:504] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:18.550928 8702 leader_election.cc:288] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [CANDIDATE]: Term 1 election: Requested vote from peers ca9e4be9fc804074a3d0d02d481b3b38 (127.29.84.129:33915), 2b9c210f183840ec93c1270003921c37 (127.29.84.131:33215)
I20260806 00:44:18.551240 8488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e995c0ebb5e4ef790067ffe1a65f33f" candidate_uuid: "be43b3654b7a495f8ab51fd621037e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9c210f183840ec93c1270003921c37"
I20260806 00:44:18.551263 8070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e995c0ebb5e4ef790067ffe1a65f33f" candidate_uuid: "be43b3654b7a495f8ab51fd621037e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca9e4be9fc804074a3d0d02d481b3b38"
I20260806 00:44:18.551359 8488 raft_consensus.cc:3037] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:18.551376 8070 raft_consensus.cc:3037] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:18.552156 8488 raft_consensus.cc:2454] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be43b3654b7a495f8ab51fd621037e66 in term 1.
I20260806 00:44:18.552217 8070 raft_consensus.cc:2454] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be43b3654b7a495f8ab51fd621037e66 in term 1.
I20260806 00:44:18.552481 8174 leader_election.cc:302] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9c210f183840ec93c1270003921c37, be43b3654b7a495f8ab51fd621037e66; no voters:
I20260806 00:44:18.552657 8702 raft_consensus.cc:2781] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:18.552834 8702 raft_consensus.cc:686] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 1 LEADER]: Becoming Leader. State: Replica: be43b3654b7a495f8ab51fd621037e66, State: Running, Role: LEADER
I20260806 00:44:18.552983 8702 consensus_queue.cc:237] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } }
I20260806 00:44:18.553416 8703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.553195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.130:33985)
0806 00:44:18.553282 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.553362 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 00:44:18.554301 7869 catalog_manager.cc:5495] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 reported cstate change: term changed from 0 to 1, leader changed from <none> to be43b3654b7a495f8ab51fd621037e66 (127.29.84.130). New cstate: current_term: 1 leader_uuid: "be43b3654b7a495f8ab51fd621037e66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be43b3654b7a495f8ab51fd621037e66" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35959 } health_report { overall_health: HEALTHY } } }
I20260806 00:44:18.570478 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.567467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44904 (local address 127.29.84.130:35959)
0806 00:44:18.567585 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:18.567591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:18.567607 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:44:18.567762 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:18.567765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:18.567813 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:18.567892 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:18.568150 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.568630 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.569491 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:18.569710 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:18.569820 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:18.570084 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:18.570152 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:18.570231 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:18.570322 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 00:44:18.570368 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:44:18.572306 8070 raft_consensus.cc:1261] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [term 1 FOLLOWER]: Refusing update from remote peer be43b3654b7a495f8ab51fd621037e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:18.572301 8488 raft_consensus.cc:1261] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [term 1 FOLLOWER]: Refusing update from remote peer be43b3654b7a495f8ab51fd621037e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:18.572768 8711 consensus_queue.cc:1035] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca9e4be9fc804074a3d0d02d481b3b38" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:18.572906 8702 consensus_queue.cc:1035] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9c210f183840ec93c1270003921c37" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 33215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:18.574357 8708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.574091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.129:50431)
0806 00:44:18.574243 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.574292 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 00:44:18.574478 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.574237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.131:58589)
0806 00:44:18.574371 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.574417 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 00:44:18.576910 8703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.576690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.130:34429)
0806 00:44:18.576808 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.576855 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 00:44:18.578692 8703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.578441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.130:49679)
0806 00:44:18.578581 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.578629 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 00:44:18.579006 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.578661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.131:56215)
0806 00:44:18.578900 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.578949 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260806 00:44:18.580117 8708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.579869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44055 (local address 127.29.84.129:42903)
0806 00:44:18.580006 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.580055 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 00:44:18.583281 7861 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:18.583459 7861 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:18.586588 7861 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70e92ce79c3941baba1206d0232d7a94: Sending DeleteTablet for 3 replicas of tablet 5e995c0ebb5e4ef790067ffe1a65f33f
I20260806 00:44:18.587296 8248 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e995c0ebb5e4ef790067ffe1a65f33f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:18 UTC) from {username='oryx', principal='oryx/127.29.84.188@KRBTEST.COM'} at 127.0.0.1:44896
I20260806 00:44:18.587287 8462 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e995c0ebb5e4ef790067ffe1a65f33f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:18 UTC) from {username='oryx', principal='oryx/127.29.84.188@KRBTEST.COM'} at 127.0.0.1:45456
I20260806 00:44:18.587378 8045 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e995c0ebb5e4ef790067ffe1a65f33f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:18 UTC) from {username='oryx', principal='oryx/127.29.84.188@KRBTEST.COM'} at 127.0.0.1:55268
I20260806 00:44:18.587566 8730 tablet_replica.cc:330] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: stopping tablet replica
I20260806 00:44:18.587595 8732 tablet_replica.cc:330] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: stopping tablet replica
I20260806 00:44:18.587692 8730 raft_consensus.cc:2229] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:44:18.587692 8732 raft_consensus.cc:2229] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:18.587796 8732 raft_consensus.cc:2258] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:18.587810 8733 tablet_replica.cc:330] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38: stopping tablet replica
I20260806 00:44:18.587847 8730 raft_consensus.cc:2258] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:18.587917 8733 raft_consensus.cc:2229] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:18.588148 8733 raft_consensus.cc:2258] T 5e995c0ebb5e4ef790067ffe1a65f33f P ca9e4be9fc804074a3d0d02d481b3b38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:18.588387 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 7806
I20260806 00:44:18.597932 8730 ts_tablet_manager.cc:1905] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:18.599947 8732 ts_tablet_manager.cc:1905] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:18.601048 8730 ts_tablet_manager.cc:1918] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:18.601138 8730 log.cc:1196] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/5e995c0ebb5e4ef790067ffe1a65f33f
I20260806 00:44:18.601485 8730 ts_tablet_manager.cc:1939] T 5e995c0ebb5e4ef790067ffe1a65f33f P be43b3654b7a495f8ab51fd621037e66: Deleting consensus metadata
I20260806 00:44:18.602210 7847 catalog_manager.cc:4841] TS be43b3654b7a495f8ab51fd621037e66 (127.29.84.130:35959): tablet 5e995c0ebb5e4ef790067ffe1a65f33f (table test-table [id=6d2346a6f984480e922c77d30f9b9f4a]) successfully deleted
I20260806 00:44:18.602550 8732 ts_tablet_manager.cc:1918] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:18.602615 8732 log.cc:1196] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/5e995c0ebb5e4ef790067ffe1a65f33f
I20260806 00:44:18.602931 8732 ts_tablet_manager.cc:1939] T 5e995c0ebb5e4ef790067ffe1a65f33f P 2b9c210f183840ec93c1270003921c37: Deleting consensus metadata
I20260806 00:44:18.603670 7846 catalog_manager.cc:4841] TS 2b9c210f183840ec93c1270003921c37 (127.29.84.131:33215): tablet 5e995c0ebb5e4ef790067ffe1a65f33f (table test-table [id=6d2346a6f984480e922c77d30f9b9f4a]) successfully deleted
I20260806 00:44:18.606268 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:18.606027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:33915 (local address 127.0.0.1:55276)
0806 00:44:18.606153 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:18.606206 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:44:18.606444 7848 catalog_manager.cc:4584] TS ca9e4be9fc804074a3d0d02d481b3b38 (127.29.84.129:33915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e995c0ebb5e4ef790067ffe1a65f33f: Network error: Client connection negotiation failed: client connection to 127.29.84.129:33915: connect: Connection refused (error 111)
I20260806 00:44:18.606496 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 8152
I20260806 00:44:18.618027 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 8356
I20260806 00:44:18.628120 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 7388
I20260806 00:44:18.642875 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 7540
2026-08-06T00:44:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5415 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:18 dist-test-slave-n326 krb5kdc[8738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:18 dist-test-slave-n326 krb5kdc[8738](info): set up 2 sockets
Aug 06 00:44:18 dist-test-slave-n326 krb5kdc[8738](info): commencing operation
krb5kdc: starting...
W20260806 00:44:19.857650 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:19 dist-test-slave-n326 krb5kdc[8738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.190.keytab.
Entry for principal oryx/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.190.keytab.
I20260806 00:44:19.972087 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:44985
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.190.keytab
--principal=oryx/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:44985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:20.076113 8760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:20.076311 8760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:20.076345 8760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:20.080238 8760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:20.080297 8760 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 00:44:20.080314 8760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:20.080332 8760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:20.080386 8760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:20.080415 8760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:20.085381 8760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:44985
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.190.keytab
--principal=oryx/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:44985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:20.086253 8760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:20.087203 8760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:20.094820 8768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:20.094820 8770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:20.094908 8766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:20.094913 8760 server_base.cc:1034] running on GCE node
I20260806 00:44:20.095793 8760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:20.096712 8760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:20.097896 8760 hybrid_clock.cc:648] HybridClock initialized: now 1785977060097876 us; error 45 us; skew 500 ppm
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:20.100641 8760 init.cc:377] Logged in from keytab as oryx/127.29.84.190@KRBTEST.COM (short username oryx)
I20260806 00:44:20.101473 8760 webserver.cc:460] Webserver started at http://127.29.84.190:35811/ using document root <none> and password file <none>
I20260806 00:44:20.101992 8760 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:20.102082 8760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:20.102298 8760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:20.104128 8760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "bb3054a78d1c4c27950d41e1602bacb6"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "cfefd673a1a378a010ca9574b42b7ba7"
server_key_iv: "ece60d2e93a49ecccfc9b7958cac979e"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.104665 8760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "bb3054a78d1c4c27950d41e1602bacb6"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "cfefd673a1a378a010ca9574b42b7ba7"
server_key_iv: "ece60d2e93a49ecccfc9b7958cac979e"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.107971 8760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:20.110160 8779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:20.111250 8760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:20.111379 8760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "bb3054a78d1c4c27950d41e1602bacb6"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "cfefd673a1a378a010ca9574b42b7ba7"
server_key_iv: "ece60d2e93a49ecccfc9b7958cac979e"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.111482 8760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:20.136255 8760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:20.138717 8760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:20.138904 8760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:20.146108 8760 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:44985
I20260806 00:44:20.146119 8868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:44985 every 8 connection(s)
I20260806 00:44:20.146916 8760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:20.147441 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 8760
I20260806 00:44:20.147604 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:20.147884 30034 external_mini_cluster.cc:1428] Setting key e5c5fc598b89528a3ae0bf5e9e01518d
I20260806 00:44:20.149946 8870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:20.156262 8870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6: Bootstrap starting.
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:20.158764 8870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:20.160080 8870 log.cc:824] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:20.162081 8870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6: No bootstrap required, opened a new log
I20260806 00:44:20.162151 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.149220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.29.84.190:44985)
0806 00:44:20.149634 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.149653 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.149695 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:44:20.150430 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.150451 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.150829 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.151226 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:20.151883 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.153352 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.154161 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.154829 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.154983 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.157369 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:20.157378 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:20.157393 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:20.157422 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:20.159445 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.159845 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.159850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.159855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.159994 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.160252 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.160255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.160257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.160637 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:20.160817 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.161030 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.161285 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":112,"threads_started":1}
I20260806 00:44:20.165158 8870 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb3054a78d1c4c27950d41e1602bacb6" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44985 } }
I20260806 00:44:20.165344 8870 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:20.165378 8870 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb3054a78d1c4c27950d41e1602bacb6, State: Initialized, Role: FOLLOWER
I20260806 00:44:20.165741 8870 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb3054a78d1c4c27950d41e1602bacb6" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44985 } }
I20260806 00:44:20.165850 8870 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:20.165895 8870 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:20.165957 8870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:20.166643 8870 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb3054a78d1c4c27950d41e1602bacb6" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44985 } }
I20260806 00:44:20.166896 8870 leader_election.cc:302] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb3054a78d1c4c27950d41e1602bacb6; no voters:
I20260806 00:44:20.167173 8870 leader_election.cc:288] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:20.167286 8875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:20.167542 8875 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [term 1 LEADER]: Becoming Leader. State: Replica: bb3054a78d1c4c27950d41e1602bacb6, State: Running, Role: LEADER
I20260806 00:44:20.167869 8875 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb3054a78d1c4c27950d41e1602bacb6" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44985 } }
I20260806 00:44:20.168344 8870 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:20.169708 8877 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb3054a78d1c4c27950d41e1602bacb6. Latest consensus state: current_term: 1 leader_uuid: "bb3054a78d1c4c27950d41e1602bacb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb3054a78d1c4c27950d41e1602bacb6" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44985 } } }
I20260806 00:44:20.169837 8877 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:20.170200 8876 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb3054a78d1c4c27950d41e1602bacb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb3054a78d1c4c27950d41e1602bacb6" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44985 } } }
I20260806 00:44:20.170308 8876 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:20.170256 8888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:20.173532 8888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:20.178514 8888 catalog_manager.cc:1347] Generated new cluster ID: b3b93aac06544b73b241d5eca5cd8c23
I20260806 00:44:20.178575 8888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:20.185240 8888 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:20.186316 8888 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:20.208508 8888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6: Generated new TSK 0
I20260806 00:44:20.209290 8888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab.
Entry for principal oryx/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab.
I20260806 00:44:20.242300 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:44985
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab
--principal=oryx/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:20.349680 8905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:20.349951 8905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:20.350028 8905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:20.353909 8905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:20.354007 8905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:20.354076 8905 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:44:20.354154 8905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:20.359638 8905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab
--principal=oryx/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.84.190:44985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:20.360515 8905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:20.361479 8905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:20.370684 8918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:20.370690 8916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:20.370682 8912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:20.371053 8905 server_base.cc:1034] running on GCE node
I20260806 00:44:20.371488 8905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:20.372074 8905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:20.373291 8905 hybrid_clock.cc:648] HybridClock initialized: now 1785977060373276 us; error 24 us; skew 500 ppm
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:20.376096 8905 init.cc:377] Logged in from keytab as oryx/127.29.84.129@KRBTEST.COM (short username oryx)
I20260806 00:44:20.376881 8905 webserver.cc:460] Webserver started at http://127.29.84.129:35039/ using document root <none> and password file <none>
I20260806 00:44:20.377396 8905 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:20.377487 8905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:20.377732 8905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:20.379691 8905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "29518182f2e34da2822f61d066619756"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "d0e1c70eed987cb8120ac09787ff2f47"
server_key_iv: "7969671d27cf738a61a72991aab66642"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.380223 8905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "29518182f2e34da2822f61d066619756"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "d0e1c70eed987cb8120ac09787ff2f47"
server_key_iv: "7969671d27cf738a61a72991aab66642"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.383761 8905 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:44:20.385958 8931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:20.387190 8905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 00:44:20.387316 8905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "29518182f2e34da2822f61d066619756"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "d0e1c70eed987cb8120ac09787ff2f47"
server_key_iv: "7969671d27cf738a61a72991aab66642"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.387450 8905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:20.402794 8905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:20.405893 8905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:20.406078 8905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:20.407521 8905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:20.407595 8905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:20.407661 8905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:20.407716 8905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:20.418594 8905 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:43893
I20260806 00:44:20.418677 9090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:43893 every 8 connection(s)
I20260806 00:44:20.419653 8905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:20.422361 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.408201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:41139 (local address 127.29.84.190:44985)
0806 00:44:20.408367 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.408374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.409624 (+ 1250us) server_negotiation.cc:407] Connection header received
0806 00:44:20.410707 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.410710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.410768 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.410855 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:20.412491 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.413384 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.415317 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.415607 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.415692 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.418772 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:20.418776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:20.418779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:20.418806 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:20.420373 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.421000 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.421003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.421004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.421088 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.421405 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.421408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.421409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.421603 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:20.421682 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.422124 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.422237 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:44:20.423195 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.409019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44985 (local address 127.29.84.129:41139)
0806 00:44:20.409449 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:20.409499 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:20.410478 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:20.411114 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:20.411126 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:20.411601 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:20.412307 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.412329 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.413534 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.413538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.415208 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.415218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.415780 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.415787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.416004 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.416651 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:20.416675 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:20.418545 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:20.420510 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.420516 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.420529 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.420882 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.421176 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.421178 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.421180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.421305 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.421693 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:20.421698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:20.421900 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 00:44:20.422049 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 00:44:20.422276 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":130,"threads_started":1}
I20260806 00:44:20.424463 9091 heartbeater.cc:347] Connected to a master server at 127.29.84.190:44985
I20260806 00:44:20.424615 9091 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:20.425104 9091 heartbeater.cc:511] Master 127.29.84.190:44985 requested a full tablet report, sending...
I20260806 00:44:20.426718 8800 ts_manager.cc:194] Registered new tserver with Master: 29518182f2e34da2822f61d066619756 (127.29.84.129:43893)
I20260806 00:44:20.428143 8800 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:41139
I20260806 00:44:20.428500 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 8905
I20260806 00:44:20.428650 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:20.429021 30034 external_mini_cluster.cc:1428] Setting key facbed24c7b256923820eabdadd5056d
I20260806 00:44:20.434207 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.426647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:39245 (local address 127.29.84.190:44985)
0806 00:44:20.426778 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.426782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.427029 (+ 247us) server_negotiation.cc:407] Connection header received
0806 00:44:20.427173 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.427175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.427214 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.427278 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:20.427748 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.428310 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.429356 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.429644 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.429741 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.430944 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:20.430948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:20.430951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:20.430974 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:20.432603 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.433010 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.433016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.433020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.433114 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.433372 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.433375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.433376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.433622 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:20.433732 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.434010 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.434104 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:44:20.434403 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.426529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44985 (local address 127.29.84.129:39245)
0806 00:44:20.426938 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:20.426952 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:20.427051 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:20.427319 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:20.427322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:20.427522 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:20.427634 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.427641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.428449 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.428452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.429230 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.429241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.429850 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.429854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.429955 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.430459 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:20.430478 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:20.430810 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:20.432723 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.432726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.432727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.432913 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.433209 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.433211 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.433213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.433284 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.433719 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:20.433721 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:20.433812 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 00:44:20.433930 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 00:44:20.434088 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for oryx/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab.
Entry for principal oryx/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab.
I20260806 00:44:20.456032 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:44985
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab
--principal=oryx/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:20.563452 9106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:20.563654 9106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:20.563694 9106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:20.567543 9106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:20.567590 9106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:20.567628 9106 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:44:20.567659 9106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:20.572821 9106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab
--principal=oryx/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.84.190:44985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:20.573633 9106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:20.574492 9106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:20.583542 9113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:20.583573 9106 server_base.cc:1034] running on GCE node
W20260806 00:44:20.583540 9111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:20.583567 9116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:20.584174 9106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:20.584733 9106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:20.585886 9106 hybrid_clock.cc:648] HybridClock initialized: now 1785977060585861 us; error 37 us; skew 500 ppm
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:20.588567 9106 init.cc:377] Logged in from keytab as oryx/127.29.84.130@KRBTEST.COM (short username oryx)
I20260806 00:44:20.589411 9106 webserver.cc:460] Webserver started at http://127.29.84.130:38341/ using document root <none> and password file <none>
I20260806 00:44:20.589946 9106 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:20.590039 9106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:20.590269 9106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:20.592268 9106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "3e340a404fdd4482a28d20d5c34969cb"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "99cc3e9a5034305f93f3e470e8b08e85"
server_key_iv: "723af86f036ee0c8d0aa8fdfe8bb729a"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.592794 9106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "3e340a404fdd4482a28d20d5c34969cb"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "99cc3e9a5034305f93f3e470e8b08e85"
server_key_iv: "723af86f036ee0c8d0aa8fdfe8bb729a"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.596138 9106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:20.598227 9125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:20.599396 9106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:20.599519 9106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "3e340a404fdd4482a28d20d5c34969cb"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "99cc3e9a5034305f93f3e470e8b08e85"
server_key_iv: "723af86f036ee0c8d0aa8fdfe8bb729a"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.599619 9106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:20.615837 9106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:20.618378 9106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:20.618569 9106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:20.620136 9106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:20.620198 9106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:20.620246 9106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:20.620276 9106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:20.632236 9106 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:46309
I20260806 00:44:20.632280 9296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:46309 every 8 connection(s)
I20260806 00:44:20.633256 9106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:20.633714 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.620738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:44871 (local address 127.29.84.190:44985)
0806 00:44:20.620882 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.620887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.621706 (+ 819us) server_negotiation.cc:407] Connection header received
0806 00:44:20.622600 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.622606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.622664 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.622769 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:20.624084 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.624825 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.626111 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.626326 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.626444 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.629705 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:20.629709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:20.629711 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:20.629735 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:20.631521 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.632073 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.632076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.632078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.632182 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.632574 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.632577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.632579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.632795 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:20.632910 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.633398 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.633521 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:44:20.634435 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.621105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44985 (local address 127.29.84.130:44871)
0806 00:44:20.621550 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:20.621597 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:20.622396 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:20.623041 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:20.623049 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:20.623492 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:20.623935 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.623947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.624999 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.625001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.625986 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.625995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.626474 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.626478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.626645 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.627210 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:20.627225 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:20.629541 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:20.631650 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.631654 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.631663 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.631952 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.632285 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.632290 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.632295 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.632473 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.632905 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:20.632910 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:20.633155 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 00:44:20.633310 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 00:44:20.633578 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":128,"threads_started":1}
I20260806 00:44:20.635478 9298 heartbeater.cc:347] Connected to a master server at 127.29.84.190:44985
I20260806 00:44:20.635660 9298 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:20.636122 9298 heartbeater.cc:511] Master 127.29.84.190:44985 requested a full tablet report, sending...
I20260806 00:44:20.637166 8801 ts_manager.cc:194] Registered new tserver with Master: 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309)
I20260806 00:44:20.637851 8801 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:44871
I20260806 00:44:20.641790 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 9106
I20260806 00:44:20.641948 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:20.642330 30034 external_mini_cluster.cc:1428] Setting key b3e614b07a1e1a75b9d9ce5ac29aa4af
I20260806 00:44:20.644411 9307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.637450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44985 (local address 127.29.84.130:50719)
0806 00:44:20.637715 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:20.637728 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:20.637842 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:20.638081 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:20.638084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:20.638280 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:20.638368 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.638373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.639114 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.639118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.639729 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.639737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.640207 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.640211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.640309 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.640715 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:20.640726 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:20.641083 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:20.642911 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.642915 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.642917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.643198 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.643459 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.643462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.643463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.643559 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.644039 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:20.644041 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:20.644114 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 00:44:20.644199 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 00:44:20.644293 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":18,"thread_start_us":86,"threads_started":1}
I20260806 00:44:20.644622 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.637539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:50719 (local address 127.29.84.190:44985)
0806 00:44:20.637679 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.637683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.637803 (+ 120us) server_negotiation.cc:407] Connection header received
0806 00:44:20.637918 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.637929 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.637995 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.638065 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:20.638487 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.638968 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.639866 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.640088 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.640194 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.641231 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:20.641234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:20.641236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:20.641257 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:20.642798 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.643286 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.643289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.643290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.643362 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.643673 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.643677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.643679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.643932 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:20.644035 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.644290 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.644412 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
WARNING: no policy specified for oryx/127.29.84.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab.
Entry for principal oryx/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab.
I20260806 00:44:20.670151 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:44985
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab
--principal=oryx/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:20.775614 9311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:20.775816 9311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:20.775884 9311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:20.779620 9311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:20.779666 9311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:20.779700 9311 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:44:20.779767 9311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:20.784698 9311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab
--principal=oryx/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.84.190:44985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:20.785580 9311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:20.786501 9311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:20.795711 9320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:20.795794 9325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:20.795881 9311 server_base.cc:1034] running on GCE node
W20260806 00:44:20.795722 9323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:20.796403 9311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:20.796967 9311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:20.798171 9311 hybrid_clock.cc:648] HybridClock initialized: now 1785977060798124 us; error 40 us; skew 500 ppm
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:20.800892 9311 init.cc:377] Logged in from keytab as oryx/127.29.84.131@KRBTEST.COM (short username oryx)
I20260806 00:44:20.801688 9311 webserver.cc:460] Webserver started at http://127.29.84.131:37821/ using document root <none> and password file <none>
I20260806 00:44:20.802208 9311 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:20.802299 9311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:20.802523 9311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:20.804503 9311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "a9a1e802045d4e5db9c06203bb71b400"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "f9240cb77e7de885e610cbf8b693b252"
server_key_iv: "6c8b3e8d7074c7d7db9cf20bcb869688"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.805032 9311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "a9a1e802045d4e5db9c06203bb71b400"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "f9240cb77e7de885e610cbf8b693b252"
server_key_iv: "6c8b3e8d7074c7d7db9cf20bcb869688"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.808429 9311 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:20.810649 9334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:20.811843 9311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:20.811976 9311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "a9a1e802045d4e5db9c06203bb71b400"
format_stamp: "Formatted at 2026-08-06 00:44:20 on dist-test-slave-n326"
server_key: "f9240cb77e7de885e610cbf8b693b252"
server_key_iv: "6c8b3e8d7074c7d7db9cf20bcb869688"
server_key_version: "encryptionkey@0"
I20260806 00:44:20.812083 9311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:20.835888 9311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:20.838963 9311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:20.839183 9311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:20.840484 9311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:20.840557 9311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:20.840624 9311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:20.840669 9311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:20.851857 9311 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:45169
I20260806 00:44:20.852569 9510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:45169 every 8 connection(s)
I20260806 00:44:20.852739 9311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:20.854432 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.841340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:54351 (local address 127.29.84.190:44985)
0806 00:44:20.841502 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.841506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.842496 (+ 990us) server_negotiation.cc:407] Connection header received
0806 00:44:20.843535 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.843539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.843598 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.843661 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:20.845287 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.846167 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.847792 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.848028 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.848146 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.850887 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:20.850890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:20.850892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:20.850917 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:20.852455 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.853028 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.853031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.853032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.853147 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.853512 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.853516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.853517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.853734 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:20.853818 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.854222 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.854320 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 00:44:20.855134 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.841876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44985 (local address 127.29.84.131:54351)
0806 00:44:20.842343 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:20.842382 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:20.843284 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:20.844096 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:20.844102 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:20.844542 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:20.845129 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.845151 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.846300 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.846304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.847668 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.847677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.848203 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.848207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.848372 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.848859 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:20.848874 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:20.850711 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:20.852572 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.852577 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.852590 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.852929 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.853259 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.853262 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.853264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.853418 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.853828 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:20.853833 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:20.854026 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 00:44:20.854170 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 00:44:20.854373 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":129,"threads_started":1}
I20260806 00:44:20.855333 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 9311
I20260806 00:44:20.855469 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:20.855827 30034 external_mini_cluster.cc:1428] Setting key d30e269d5457c2afcc3ae1d29cb99878
I20260806 00:44:20.856456 9511 heartbeater.cc:347] Connected to a master server at 127.29.84.190:44985
I20260806 00:44:20.856617 9511 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:20.857115 9511 heartbeater.cc:511] Master 127.29.84.190:44985 requested a full tablet report, sending...
I20260806 00:44:20.858040 8800 ts_manager.cc:194] Registered new tserver with Master: a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169)
I20260806 00:44:20.858806 8800 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:54351
I20260806 00:44:20.859530 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:44:20.865268 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.858467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:46123 (local address 127.29.84.190:44985)
0806 00:44:20.858563 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.858567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.858906 (+ 339us) server_negotiation.cc:407] Connection header received
0806 00:44:20.859034 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.859036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.859104 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.859198 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:20.859686 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.860154 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.860955 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.861179 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.861285 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.862266 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:20.862270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:20.862272 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:20.862295 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:20.863675 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.864082 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.864087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.864089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.864166 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.864523 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.864526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.864527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.864688 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:20.864793 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.865042 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.865143 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:44:20.865284 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.858406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44985 (local address 127.29.84.131:46123)
0806 00:44:20.858814 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:20.858825 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:20.858935 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:20.859212 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:20.859215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:20.859439 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:20.859545 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.859554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.860276 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.860279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.860832 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.860839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.861289 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.861293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.861400 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.861857 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:20.861867 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:20.862137 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:20.863783 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.863787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.863789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.863984 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.864261 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:20.864265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:20.864267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:20.864384 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:20.864785 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:20.864788 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:20.864880 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 00:44:20.864987 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 00:44:20.865113 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":91,"threads_started":1}
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:44:20.884136 8873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:20.878707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45192 (local address 127.29.84.190:44985)
0806 00:44:20.878817 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.878820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.878831 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:44:20.878859 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.878861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.878891 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.878955 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:20.879659 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.880102 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.880902 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.881095 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.881166 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.884052 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45192: BlockingRecv error: recv got EOF from 127.0.0.1:45192 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 00:44:20 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:20.892902 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.885416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.29.84.190:44985)
0806 00:44:20.885674 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.885677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.885690 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:44:20.885717 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.885719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.885748 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.885825 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:20.886343 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.886821 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.887575 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.887807 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.887879 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.889900 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:20.889910 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:20.889913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:20.889933 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:20.891335 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.891773 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.891776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.891778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.891872 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:20.892214 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:20.892219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:20.892220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:20.892418 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:20.892523 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.892721 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.892804 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260806 00:44:20.895499 8801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:44:20.898221 8801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:20.912292 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.906822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.29.84.131:45169)
0806 00:44:20.907602 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.907611 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.907627 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:44:20.907678 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.907681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.907827 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.907979 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:20.908274 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.909410 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.910807 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.911619 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.911736 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.911884 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.912063 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.912154 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":103,"threads_started":1}
I20260806 00:44:20.912304 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.906457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45169 (local address 127.0.0.1:53532)
0806 00:44:20.907302 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:20.907316 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:20.907434 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:20.907958 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:20.907961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:20.907978 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:20.908146 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.908153 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.909546 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.909550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.910683 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.910695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.911739 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.911742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.911860 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.911914 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 00:44:20.912029 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 00:44:20.912153 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":90,"thread_start_us":82,"threads_started":1}
I20260806 00:44:20.912510 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.906856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41324 (local address 127.29.84.129:43893)
0806 00:44:20.907451 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.907469 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.907496 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:44:20.907577 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.907581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.907723 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.907912 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:20.908184 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.909064 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.910712 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.911366 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.911493 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.911650 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.912251 (+ 601us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.912347 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":70,"threads_started":1}
I20260806 00:44:20.912748 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.906227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:43893 (local address 127.0.0.1:41324)
0806 00:44:20.907292 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:20.907305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:20.907396 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:20.907891 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:20.907897 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:20.907920 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:20.908071 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.908079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.909165 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.909171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.910580 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.910591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.911729 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.911733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.911854 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.911907 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 00:44:20.912017 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 00:44:20.912138 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":72,"thread_start_us":78,"threads_started":1}
I20260806 00:44:20.913530 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.906139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46309 (local address 127.0.0.1:39608)
0806 00:44:20.906849 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:20.906896 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:20.907007 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:20.908666 (+ 1659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:20.908670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:20.908690 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:20.908781 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.908786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.910226 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.910231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.911348 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:20.911359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.912734 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.912737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:20.913245 (+ 508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.913265 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:44:20.913325 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 00:44:20.913398 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":4,"thread_start_us":68,"threads_started":1}
I20260806 00:44:20.914145 9543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:20.906263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39608 (local address 127.29.84.130:46309)
0806 00:44:20.908231 (+ 1968us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:20.908244 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:20.908266 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:44:20.908339 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:20.908346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:20.908519 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:20.908666 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:20.908911 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.910110 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.911478 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:20.912228 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:20.912325 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:20.913882 (+ 1557us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:20.913928 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 00:44:20.914034 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1850,"thread_start_us":106,"threads_started":1}
I20260806 00:44:20.915166 8992 tablet_service.cc:1467] Processing CreateTablet for tablet 02dd9bce31544dfda05b3fbbbb222b7c (DEFAULT_TABLE table=test-table [id=4980dbd3fdd74153881ad34d10ac6847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:20.915158 9417 tablet_service.cc:1467] Processing CreateTablet for tablet 02dd9bce31544dfda05b3fbbbb222b7c (DEFAULT_TABLE table=test-table [id=4980dbd3fdd74153881ad34d10ac6847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:20.916206 8992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02dd9bce31544dfda05b3fbbbb222b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:20.916206 9417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02dd9bce31544dfda05b3fbbbb222b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:20.916465 9191 tablet_service.cc:1467] Processing CreateTablet for tablet 02dd9bce31544dfda05b3fbbbb222b7c (DEFAULT_TABLE table=test-table [id=4980dbd3fdd74153881ad34d10ac6847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:20.917327 9191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02dd9bce31544dfda05b3fbbbb222b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:20.923323 9550 tablet_bootstrap.cc:492] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: Bootstrap starting.
I20260806 00:44:20.924232 9551 tablet_bootstrap.cc:492] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: Bootstrap starting.
I20260806 00:44:20.926018 9551 tablet_bootstrap.cc:654] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:20.926034 9550 tablet_bootstrap.cc:654] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:20.926728 9551 log.cc:824] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:20.926728 9550 log.cc:824] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:20.927155 9552 tablet_bootstrap.cc:492] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: Bootstrap starting.
I20260806 00:44:20.928334 9551 tablet_bootstrap.cc:492] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: No bootstrap required, opened a new log
I20260806 00:44:20.928537 9551 ts_tablet_manager.cc:1397] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 00:44:20.928544 9550 tablet_bootstrap.cc:492] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: No bootstrap required, opened a new log
I20260806 00:44:20.928722 9550 ts_tablet_manager.cc:1397] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 00:44:20.929008 9552 tablet_bootstrap.cc:654] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:20.929782 9552 log.cc:824] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:20.931733 9552 tablet_bootstrap.cc:492] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: No bootstrap required, opened a new log
I20260806 00:44:20.931902 9552 ts_tablet_manager.cc:1397] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 00:44:20.932137 9550 raft_consensus.cc:348] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:20.932171 9551 raft_consensus.cc:348] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:20.932345 9550 raft_consensus.cc:374] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:20.932394 9551 raft_consensus.cc:374] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:20.932399 9550 raft_consensus.cc:729] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e340a404fdd4482a28d20d5c34969cb, State: Initialized, Role: FOLLOWER
I20260806 00:44:20.932452 9551 raft_consensus.cc:729] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29518182f2e34da2822f61d066619756, State: Initialized, Role: FOLLOWER
I20260806 00:44:20.932829 9550 consensus_queue.cc:260] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:20.932938 9551 consensus_queue.cc:260] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:20.933409 9298 heartbeater.cc:503] Master 127.29.84.190:44985 was elected leader, sending a full tablet report...
I20260806 00:44:20.933812 9550 ts_tablet_manager.cc:1428] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:44:20.933871 9551 ts_tablet_manager.cc:1428] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:44:20.934183 9091 heartbeater.cc:503] Master 127.29.84.190:44985 was elected leader, sending a full tablet report...
I20260806 00:44:20.935541 9552 raft_consensus.cc:348] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:20.935724 9552 raft_consensus.cc:374] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:20.935755 9552 raft_consensus.cc:729] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a1e802045d4e5db9c06203bb71b400, State: Initialized, Role: FOLLOWER
I20260806 00:44:20.936196 9552 consensus_queue.cc:260] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:20.937386 9552 ts_tablet_manager.cc:1428] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260806 00:44:20.937764 9511 heartbeater.cc:503] Master 127.29.84.190:44985 was elected leader, sending a full tablet report...
I20260806 00:44:21.045445 9567 raft_consensus.cc:482] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:21.045650 9567 raft_consensus.cc:504] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.046682 9567 leader_election.cc:288] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309), a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169)
I20260806 00:44:21.050593 9543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.047196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:57003 (local address 127.29.84.130:46309)
0806 00:44:21.047354 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.047361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.047380 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:44:21.047570 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.047573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.047617 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.047703 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:21.048006 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.048532 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.049612 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.050016 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.050159 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.050212 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.050389 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.050463 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:44:21.050611 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.046925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46309 (local address 127.29.84.129:57003)
0806 00:44:21.047309 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.047326 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.047441 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.047718 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.047721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.047738 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:21.047873 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.047878 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.048652 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.048657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.049476 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.049486 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.050157 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.050161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.050255 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.050284 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.050371 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.050439 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":79,"threads_started":1}
I20260806 00:44:21.050616 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.047185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:51347 (local address 127.29.84.131:45169)
0806 00:44:21.047353 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.047359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.047540 (+ 181us) server_negotiation.cc:407] Connection header received
0806 00:44:21.047707 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.047712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.047757 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.047843 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:21.048113 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.048634 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.049650 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.050036 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.050181 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.050232 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.050424 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.050487 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 00:44:21.050633 9571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.046937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45169 (local address 127.29.84.129:51347)
0806 00:44:21.047444 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.047456 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.047580 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.047857 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.047860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.047878 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:21.047999 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.048004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.048753 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.048758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.049508 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.049517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.050179 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.050183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.050304 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.050321 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.050406 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.050474 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":3,"thread_start_us":134,"threads_started":1}
I20260806 00:44:21.051276 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02dd9bce31544dfda05b3fbbbb222b7c" candidate_uuid: "29518182f2e34da2822f61d066619756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e340a404fdd4482a28d20d5c34969cb" is_pre_election: true
I20260806 00:44:21.051280 9442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02dd9bce31544dfda05b3fbbbb222b7c" candidate_uuid: "29518182f2e34da2822f61d066619756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a1e802045d4e5db9c06203bb71b400" is_pre_election: true
I20260806 00:44:21.051550 9442 raft_consensus.cc:2454] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29518182f2e34da2822f61d066619756 in term 0.
I20260806 00:44:21.051558 9213 raft_consensus.cc:2454] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29518182f2e34da2822f61d066619756 in term 0.
I20260806 00:44:21.052078 8936 leader_election.cc:302] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 29518182f2e34da2822f61d066619756, 3e340a404fdd4482a28d20d5c34969cb, a9a1e802045d4e5db9c06203bb71b400; no voters:
I20260806 00:44:21.052356 9567 raft_consensus.cc:2781] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:21.052438 9567 raft_consensus.cc:482] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:21.052481 9567 raft_consensus.cc:3037] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.053534 9567 raft_consensus.cc:504] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.053841 9567 leader_election.cc:288] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [CANDIDATE]: Term 1 election: Requested vote from peers 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309), a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169)
I20260806 00:44:21.054169 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02dd9bce31544dfda05b3fbbbb222b7c" candidate_uuid: "29518182f2e34da2822f61d066619756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e340a404fdd4482a28d20d5c34969cb"
I20260806 00:44:21.054169 9442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02dd9bce31544dfda05b3fbbbb222b7c" candidate_uuid: "29518182f2e34da2822f61d066619756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a1e802045d4e5db9c06203bb71b400"
I20260806 00:44:21.054283 9442 raft_consensus.cc:3037] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.054306 9213 raft_consensus.cc:3037] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.055627 9213 raft_consensus.cc:2454] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29518182f2e34da2822f61d066619756 in term 1.
I20260806 00:44:21.055631 9442 raft_consensus.cc:2454] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29518182f2e34da2822f61d066619756 in term 1.
I20260806 00:44:21.055948 8936 leader_election.cc:302] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29518182f2e34da2822f61d066619756, a9a1e802045d4e5db9c06203bb71b400; no voters:
I20260806 00:44:21.056147 9567 raft_consensus.cc:2781] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:21.056422 9567 raft_consensus.cc:686] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 1 LEADER]: Becoming Leader. State: Replica: 29518182f2e34da2822f61d066619756, State: Running, Role: LEADER
I20260806 00:44:21.056741 9567 consensus_queue.cc:237] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.059796 8800 catalog_manager.cc:5495] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29518182f2e34da2822f61d066619756 (127.29.84.129). New cstate: current_term: 1 leader_uuid: "29518182f2e34da2822f61d066619756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } health_report { overall_health: UNKNOWN } } }
I20260806 00:44:21.099987 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.096947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.29.84.129:43893)
0806 00:44:21.097056 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.097060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.097080 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:44:21.097173 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.097175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.097218 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.097299 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:21.097500 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.097951 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.098766 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.098969 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.099043 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.099323 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:21.099697 (+ 374us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:21.099780 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.099839 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.099881 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 00:44:21.103997 9512 tablet.cc:2369] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:44:21.105603 9213 raft_consensus.cc:1261] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Refusing update from remote peer 29518182f2e34da2822f61d066619756: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:21.105609 9442 raft_consensus.cc:1261] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Refusing update from remote peer 29518182f2e34da2822f61d066619756: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:21.106333 9573 consensus_queue.cc:1035] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:21.106573 9567 consensus_queue.cc:1035] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:21.121326 8801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:21.121477 8801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:21.123792 8801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6: Sending DeleteTablet for 3 replicas of tablet 02dd9bce31544dfda05b3fbbbb222b7c
I20260806 00:44:21.124547 9191 tablet_service.cc:1514] Processing DeleteTablet for tablet 02dd9bce31544dfda05b3fbbbb222b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:21 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:39608
I20260806 00:44:21.124552 8992 tablet_service.cc:1514] Processing DeleteTablet for tablet 02dd9bce31544dfda05b3fbbbb222b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:21 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:41324
I20260806 00:44:21.124666 9417 tablet_service.cc:1514] Processing DeleteTablet for tablet 02dd9bce31544dfda05b3fbbbb222b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:21 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53532
I20260806 00:44:21.125151 9599 tablet_replica.cc:330] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: stopping tablet replica
I20260806 00:44:21.125427 9598 tablet_replica.cc:330] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: stopping tablet replica
I20260806 00:44:21.125531 9599 raft_consensus.cc:2229] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:21.125788 9598 raft_consensus.cc:2229] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:21.125787 8801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 00:44:21.126185 9598 raft_consensus.cc:2258] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 00:44:21.126185 8801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:21.126996 9596 tablet_replica.cc:330] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: stopping tablet replica
I20260806 00:44:21.127278 9596 raft_consensus.cc:2229] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:44:21.127631 9596 raft_consensus.cc:2258] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:21.127864 9598 ts_tablet_manager.cc:1905] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:21.128878 9596 ts_tablet_manager.cc:1905] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:21.130647 9598 ts_tablet_manager.cc:1918] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:21.130810 9598 log.cc:1196] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/02dd9bce31544dfda05b3fbbbb222b7c
I20260806 00:44:21.131227 9598 ts_tablet_manager.cc:1939] T 02dd9bce31544dfda05b3fbbbb222b7c P 3e340a404fdd4482a28d20d5c34969cb: Deleting consensus metadata
I20260806 00:44:21.131747 9596 ts_tablet_manager.cc:1918] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:21.131846 9596 log.cc:1196] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/02dd9bce31544dfda05b3fbbbb222b7c
I20260806 00:44:21.132175 9596 ts_tablet_manager.cc:1939] T 02dd9bce31544dfda05b3fbbbb222b7c P 29518182f2e34da2822f61d066619756: Deleting consensus metadata
I20260806 00:44:21.132632 8780 catalog_manager.cc:4841] TS 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309): tablet 02dd9bce31544dfda05b3fbbbb222b7c (table test-table [id=4980dbd3fdd74153881ad34d10ac6847]) successfully deleted
I20260806 00:44:21.133163 8783 catalog_manager.cc:4841] TS 29518182f2e34da2822f61d066619756 (127.29.84.129:43893): tablet 02dd9bce31544dfda05b3fbbbb222b7c (table test-table [id=4980dbd3fdd74153881ad34d10ac6847]) successfully deleted
I20260806 00:44:21.125860 9599 raft_consensus.cc:2258] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:21.134510 9599 ts_tablet_manager.cc:1905] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:21.134521 9191 tablet_service.cc:1467] Processing CreateTablet for tablet 0ff36909f8104251b1ef620292599149 (DEFAULT_TABLE table=test-table [id=7003b04736d54554a8deed6d93a06629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:21.134660 9417 tablet_service.cc:1467] Processing CreateTablet for tablet 0ff36909f8104251b1ef620292599149 (DEFAULT_TABLE table=test-table [id=7003b04736d54554a8deed6d93a06629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:21.134783 9191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ff36909f8104251b1ef620292599149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.134887 9417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ff36909f8104251b1ef620292599149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.136880 9550 tablet_bootstrap.cc:492] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: Bootstrap starting.
I20260806 00:44:21.137720 8992 tablet_service.cc:1467] Processing CreateTablet for tablet 0ff36909f8104251b1ef620292599149 (DEFAULT_TABLE table=test-table [id=7003b04736d54554a8deed6d93a06629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:21.137802 9550 tablet_bootstrap.cc:654] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.137980 8992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ff36909f8104251b1ef620292599149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.138374 9552 tablet_bootstrap.cc:492] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: Bootstrap starting.
I20260806 00:44:21.138957 9550 tablet_bootstrap.cc:492] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: No bootstrap required, opened a new log
I20260806 00:44:21.139036 9550 ts_tablet_manager.cc:1397] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:21.139292 9552 tablet_bootstrap.cc:654] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.139544 9550 raft_consensus.cc:348] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.139683 9550 raft_consensus.cc:374] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.139719 9550 raft_consensus.cc:729] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e340a404fdd4482a28d20d5c34969cb, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.139850 9550 consensus_queue.cc:260] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.140110 9550 ts_tablet_manager.cc:1428] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:21.140223 9599 ts_tablet_manager.cc:1918] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:21.140301 9599 log.cc:1196] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/02dd9bce31544dfda05b3fbbbb222b7c
I20260806 00:44:21.140592 9599 ts_tablet_manager.cc:1939] T 02dd9bce31544dfda05b3fbbbb222b7c P a9a1e802045d4e5db9c06203bb71b400: Deleting consensus metadata
I20260806 00:44:21.140738 9551 tablet_bootstrap.cc:492] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: Bootstrap starting.
I20260806 00:44:21.141652 9551 tablet_bootstrap.cc:654] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.142516 9552 tablet_bootstrap.cc:492] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: No bootstrap required, opened a new log
I20260806 00:44:21.142619 9552 ts_tablet_manager.cc:1397] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 00:44:21.142978 9552 raft_consensus.cc:348] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.143121 9552 raft_consensus.cc:374] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.143163 9552 raft_consensus.cc:729] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a1e802045d4e5db9c06203bb71b400, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.143218 8784 catalog_manager.cc:4841] TS a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169): tablet 02dd9bce31544dfda05b3fbbbb222b7c (table test-table [id=4980dbd3fdd74153881ad34d10ac6847]) successfully deleted
I20260806 00:44:21.143282 9552 consensus_queue.cc:260] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.143567 9552 ts_tablet_manager.cc:1428] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:21.143782 9551 tablet_bootstrap.cc:492] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: No bootstrap required, opened a new log
I20260806 00:44:21.143848 9551 ts_tablet_manager.cc:1397] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 00:44:21.144289 9551 raft_consensus.cc:348] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.144392 9551 raft_consensus.cc:374] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.144425 9551 raft_consensus.cc:729] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29518182f2e34da2822f61d066619756, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.144528 9551 consensus_queue.cc:260] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.144804 9551 ts_tablet_manager.cc:1428] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260806 00:44:21.171111 9094 tablet.cc:2369] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:44:21.317600 9566 raft_consensus.cc:482] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:21.317780 9566 raft_consensus.cc:504] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.319770 9566 leader_election.cc:288] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29518182f2e34da2822f61d066619756 (127.29.84.129:43893), a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169)
I20260806 00:44:21.326114 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.320370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:45365 (local address 127.29.84.129:43893)
0806 00:44:21.320529 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.320537 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.321068 (+ 531us) server_negotiation.cc:407] Connection header received
0806 00:44:21.321253 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.321257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.321321 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.321447 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:21.321882 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.322620 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.324650 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.325320 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.325459 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.325542 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.325858 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.325963 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:44:21.326155 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.319966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:43893 (local address 127.29.84.130:45365)
0806 00:44:21.320949 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.320965 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.321125 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.321538 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.321559 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.321589 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:21.321750 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.321760 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.322812 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.322818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.324467 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.324485 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.325470 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.325475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.325681 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.325724 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.325834 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.325931 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"thread_start_us":85,"threads_started":1}
I20260806 00:44:21.326952 9028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff36909f8104251b1ef620292599149" candidate_uuid: "3e340a404fdd4482a28d20d5c34969cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29518182f2e34da2822f61d066619756" is_pre_election: true
I20260806 00:44:21.327275 9028 raft_consensus.cc:2454] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e340a404fdd4482a28d20d5c34969cb in term 0.
I20260806 00:44:21.327860 9127 leader_election.cc:302] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29518182f2e34da2822f61d066619756, 3e340a404fdd4482a28d20d5c34969cb; no voters:
I20260806 00:44:21.328190 9566 raft_consensus.cc:2781] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:21.328260 9566 raft_consensus.cc:482] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:21.328317 9566 raft_consensus.cc:3037] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.329749 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.320610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:46227 (local address 127.29.84.131:45169)
0806 00:44:21.320754 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.320761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.322409 (+ 1648us) server_negotiation.cc:407] Connection header received
0806 00:44:21.322595 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.322598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.322649 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.322700 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:21.326398 (+ 3698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.327168 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.328381 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.329124 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.329253 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.329308 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.329580 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.329648 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:44:21.329748 9611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.320157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45169 (local address 127.29.84.130:46227)
0806 00:44:21.322324 (+ 2167us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.322338 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.322474 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.322855 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.322857 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.322874 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:21.323011 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.323018 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.327258 (+ 4240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.327264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.328221 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.328231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.329295 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.329300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.329444 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.329470 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.329548 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.329616 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2101,"thread_start_us":78,"threads_started":1}
I20260806 00:44:21.329952 9566 raft_consensus.cc:504] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.330381 9566 leader_election.cc:288] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [CANDIDATE]: Term 1 election: Requested vote from peers 29518182f2e34da2822f61d066619756 (127.29.84.129:43893), a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169)
I20260806 00:44:21.331089 9442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff36909f8104251b1ef620292599149" candidate_uuid: "3e340a404fdd4482a28d20d5c34969cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a1e802045d4e5db9c06203bb71b400" is_pre_election: true
I20260806 00:44:21.331264 9442 raft_consensus.cc:2454] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e340a404fdd4482a28d20d5c34969cb in term 0.
I20260806 00:44:21.331089 9441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff36909f8104251b1ef620292599149" candidate_uuid: "3e340a404fdd4482a28d20d5c34969cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a1e802045d4e5db9c06203bb71b400"
I20260806 00:44:21.331553 9441 raft_consensus.cc:3037] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.333056 9441 raft_consensus.cc:2454] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e340a404fdd4482a28d20d5c34969cb in term 1.
I20260806 00:44:21.333702 9129 leader_election.cc:302] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e340a404fdd4482a28d20d5c34969cb, a9a1e802045d4e5db9c06203bb71b400; no voters:
I20260806 00:44:21.333979 9028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff36909f8104251b1ef620292599149" candidate_uuid: "3e340a404fdd4482a28d20d5c34969cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29518182f2e34da2822f61d066619756"
I20260806 00:44:21.334131 9028 raft_consensus.cc:3037] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.335148 9566 raft_consensus.cc:2781] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:21.335322 9566 raft_consensus.cc:686] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 1 LEADER]: Becoming Leader. State: Replica: 3e340a404fdd4482a28d20d5c34969cb, State: Running, Role: LEADER
I20260806 00:44:21.335623 9028 raft_consensus.cc:2454] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e340a404fdd4482a28d20d5c34969cb in term 1.
I20260806 00:44:21.335587 9566 consensus_queue.cc:237] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.338325 8800 catalog_manager.cc:5495] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130). New cstate: current_term: 1 leader_uuid: "3e340a404fdd4482a28d20d5c34969cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } health_report { overall_health: UNKNOWN } } }
W20260806 00:44:21.384864 9299 tablet.cc:2369] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 00:44:21 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977060, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.190@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:21.414306 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.404790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44985 (local address 127.0.0.1:45214)
0806 00:44:21.405176 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.405196 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.405323 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.405598 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.405601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.406009 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:21.406101 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.406108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.406996 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.407002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.407660 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.407669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.408128 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.408131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.408204 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.408902 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:21.408915 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:21.410315 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:21.412361 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:21.412374 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:21.412392 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:21.412774 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:21.413077 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:21.413083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:21.413086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:21.413199 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:21.413571 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:21.413580 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:21.413752 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.413882 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.414029 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":101,"threads_started":1}
I20260806 00:44:21.414928 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.404888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45214 (local address 127.29.84.190:44985)
0806 00:44:21.405126 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.405135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.405270 (+ 135us) server_negotiation.cc:407] Connection header received
0806 00:44:21.405432 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.405435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.405483 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.405576 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:21.406223 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.406802 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.407784 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.408016 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.408126 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.410450 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:21.410454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:21.410456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:21.410476 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:21.412231 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:21.412875 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:21.412878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:21.412879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:21.412954 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:21.413299 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:21.413301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:21.413303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:21.413464 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:21.413540 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.414564 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.414698 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260806 00:44:21.419165 8800 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:45214:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 00:44:21.419885 8800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:21.425117 8992 tablet_service.cc:1467] Processing CreateTablet for tablet 4fef26fec17c40dd8e620092132ab31e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a70db887c20541fda7fb96a4838f9e9c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:44:21.425200 9191 tablet_service.cc:1467] Processing CreateTablet for tablet 4fef26fec17c40dd8e620092132ab31e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a70db887c20541fda7fb96a4838f9e9c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:44:21.425521 9191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fef26fec17c40dd8e620092132ab31e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.425522 8992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fef26fec17c40dd8e620092132ab31e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.427582 9417 tablet_service.cc:1467] Processing CreateTablet for tablet 4fef26fec17c40dd8e620092132ab31e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a70db887c20541fda7fb96a4838f9e9c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:44:21.428045 9417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fef26fec17c40dd8e620092132ab31e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.428356 9551 tablet_bootstrap.cc:492] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: Bootstrap starting.
I20260806 00:44:21.428408 9550 tablet_bootstrap.cc:492] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: Bootstrap starting.
I20260806 00:44:21.429236 9550 tablet_bootstrap.cc:654] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.429236 9551 tablet_bootstrap.cc:654] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.430621 9551 tablet_bootstrap.cc:492] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: No bootstrap required, opened a new log
I20260806 00:44:21.430680 9551 ts_tablet_manager.cc:1397] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:21.430977 9551 raft_consensus.cc:348] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.431056 9551 raft_consensus.cc:374] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.431165 9551 raft_consensus.cc:729] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29518182f2e34da2822f61d066619756, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.431300 9551 consensus_queue.cc:260] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.431563 9551 ts_tablet_manager.cc:1428] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:44:21.431697 9550 tablet_bootstrap.cc:492] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: No bootstrap required, opened a new log
I20260806 00:44:21.431778 9550 ts_tablet_manager.cc:1397] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 00:44:21.431805 9587 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.431931 9587 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:44:21.432111 9552 tablet_bootstrap.cc:492] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: Bootstrap starting.
I20260806 00:44:21.432231 9550 raft_consensus.cc:348] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.432346 9550 raft_consensus.cc:374] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.432379 9550 raft_consensus.cc:729] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e340a404fdd4482a28d20d5c34969cb, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.432490 9550 consensus_queue.cc:260] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.432760 9550 ts_tablet_manager.cc:1428] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:21.432935 9566 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.433090 9552 tablet_bootstrap.cc:654] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.433081 9566 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:44:21.434401 9552 tablet_bootstrap.cc:492] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: No bootstrap required, opened a new log
I20260806 00:44:21.434484 9552 ts_tablet_manager.cc:1397] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:21.434931 9552 raft_consensus.cc:348] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.435045 9552 raft_consensus.cc:374] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.435122 9552 raft_consensus.cc:729] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a1e802045d4e5db9c06203bb71b400, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.435252 9552 consensus_queue.cc:260] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.435534 9568 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.435660 9568 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:44:21.435961 9552 ts_tablet_manager.cc:1428] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:21.564579 9587 raft_consensus.cc:482] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:21.564770 9587 raft_consensus.cc:504] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.565204 9587 leader_election.cc:288] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309), a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169)
I20260806 00:44:21.565619 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fef26fec17c40dd8e620092132ab31e" candidate_uuid: "29518182f2e34da2822f61d066619756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e340a404fdd4482a28d20d5c34969cb" is_pre_election: true
I20260806 00:44:21.565771 9213 raft_consensus.cc:2454] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29518182f2e34da2822f61d066619756 in term 0.
I20260806 00:44:21.565657 9441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fef26fec17c40dd8e620092132ab31e" candidate_uuid: "29518182f2e34da2822f61d066619756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a1e802045d4e5db9c06203bb71b400" is_pre_election: true
I20260806 00:44:21.565972 9441 raft_consensus.cc:2454] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29518182f2e34da2822f61d066619756 in term 0.
I20260806 00:44:21.566220 8934 leader_election.cc:302] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29518182f2e34da2822f61d066619756, 3e340a404fdd4482a28d20d5c34969cb; no voters:
I20260806 00:44:21.566382 9587 raft_consensus.cc:2781] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:21.566447 9587 raft_consensus.cc:482] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:21.566488 9587 raft_consensus.cc:3037] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.567495 9587 raft_consensus.cc:504] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.567817 9587 leader_election.cc:288] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [CANDIDATE]: Term 1 election: Requested vote from peers 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309), a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169)
I20260806 00:44:21.568123 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fef26fec17c40dd8e620092132ab31e" candidate_uuid: "29518182f2e34da2822f61d066619756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e340a404fdd4482a28d20d5c34969cb"
I20260806 00:44:21.568251 9213 raft_consensus.cc:3037] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.568220 9441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fef26fec17c40dd8e620092132ab31e" candidate_uuid: "29518182f2e34da2822f61d066619756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a1e802045d4e5db9c06203bb71b400"
I20260806 00:44:21.568342 9441 raft_consensus.cc:3037] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.569168 9213 raft_consensus.cc:2454] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29518182f2e34da2822f61d066619756 in term 1.
I20260806 00:44:21.569469 9441 raft_consensus.cc:2454] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29518182f2e34da2822f61d066619756 in term 1.
I20260806 00:44:21.569546 8934 leader_election.cc:302] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29518182f2e34da2822f61d066619756, 3e340a404fdd4482a28d20d5c34969cb; no voters:
I20260806 00:44:21.569728 9587 raft_consensus.cc:2781] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:21.569832 9587 raft_consensus.cc:686] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [term 1 LEADER]: Becoming Leader. State: Replica: 29518182f2e34da2822f61d066619756, State: Running, Role: LEADER
I20260806 00:44:21.569969 9587 consensus_queue.cc:237] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.570603 9567 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: TxnStatusTablet state changed. Reason: New leader 29518182f2e34da2822f61d066619756. Latest consensus state: current_term: 1 leader_uuid: "29518182f2e34da2822f61d066619756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.570714 9567 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:44:21.570986 9629 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 00:44:21.571189 9629 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 00:44:21.571703 8801 catalog_manager.cc:5495] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29518182f2e34da2822f61d066619756 (127.29.84.129). New cstate: current_term: 1 leader_uuid: "29518182f2e34da2822f61d066619756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } health_report { overall_health: UNKNOWN } } }
I20260806 00:44:21.591162 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.587610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41352 (local address 127.29.84.129:43893)
0806 00:44:21.587738 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.587743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.587823 (+ 80us) server_negotiation.cc:407] Connection header received
0806 00:44:21.587976 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.587979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.588029 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.588110 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:21.588375 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.589138 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.589996 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.590330 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.590436 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.590709 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:21.590785 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:21.590857 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.590961 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.591010 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:44:21.591199 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.587496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:43893 (local address 127.0.0.1:41352)
0806 00:44:21.587769 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.587781 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.587873 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.588133 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.588135 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.588141 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:44:21.588273 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.588277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.589234 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.589236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.589881 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.589890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.590430 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.590433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.590502 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.590545 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:44:21.590870 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:44:21.590883 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.590949 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.591019 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260806 00:44:21.593494 9441 raft_consensus.cc:1261] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Refusing update from remote peer 29518182f2e34da2822f61d066619756: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:21.593596 9213 raft_consensus.cc:1261] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Refusing update from remote peer 29518182f2e34da2822f61d066619756: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:21.594002 9567 consensus_queue.cc:1035] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:21.594152 9587 consensus_queue.cc:1035] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:21.595464 9566 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: TxnStatusTablet state changed. Reason: New leader 29518182f2e34da2822f61d066619756. Latest consensus state: current_term: 1 leader_uuid: "29518182f2e34da2822f61d066619756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.595587 9566 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:44:21.596474 9568 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: TxnStatusTablet state changed. Reason: New leader 29518182f2e34da2822f61d066619756. Latest consensus state: current_term: 1 leader_uuid: "29518182f2e34da2822f61d066619756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.596601 9568 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:44:21.597975 9587 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29518182f2e34da2822f61d066619756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.598071 9587 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:44:21.598168 9587 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29518182f2e34da2822f61d066619756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.598227 9587 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P 29518182f2e34da2822f61d066619756: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:44:21.598364 9579 mvcc.cc:205] Tried to move back new op lower bound from 7315362044283224064 to 7315362044191682560. Current Snapshot: MvccSnapshot[applied={T|T < 7315362044283224064 or (T in {7315362044283224064})}]
I20260806 00:44:21.599192 9581 mvcc.cc:205] Tried to move back new op lower bound from 7315362044283224064 to 7315362044191682560. Current Snapshot: MvccSnapshot[applied={T|T < 7315362044283224064}]
I20260806 00:44:21.599376 9580 mvcc.cc:205] Tried to move back new op lower bound from 7315362044283224064 to 7315362044191682560. Current Snapshot: MvccSnapshot[applied={T|T < 7315362044283224064}]
I20260806 00:44:21.599539 9566 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29518182f2e34da2822f61d066619756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.599628 9566 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P 3e340a404fdd4482a28d20d5c34969cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:44:21.601300 9568 tablet_replica.cc:439] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29518182f2e34da2822f61d066619756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } } }
I20260806 00:44:21.601433 9568 tablet_replica.cc:442] T 4fef26fec17c40dd8e620092132ab31e P a9a1e802045d4e5db9c06203bb71b400: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:44:21.607596 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.603315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.29.84.130:46309)
0806 00:44:21.603579 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.603586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.603695 (+ 109us) server_negotiation.cc:407] Connection header received
0806 00:44:21.603864 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.603870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.603952 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.604042 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:21.604344 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.605180 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.606020 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.606377 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.606479 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.606698 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:21.607208 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:21.607311 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.607373 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.607425 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260806 00:44:21.619019 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.614023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:36943 (local address 127.29.84.129:43893)
0806 00:44:21.614143 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.614147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.615378 (+ 1231us) server_negotiation.cc:407] Connection header received
0806 00:44:21.615503 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.615506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.615559 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.615626 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:21.615877 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.616439 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.618069 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.618271 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.618342 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.618655 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:21.618713 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:21.618765 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.618887 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.618929 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:44:21.619143 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.613936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:43893 (local address 127.29.84.130:36943)
0806 00:44:21.615279 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.615294 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.615404 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.615660 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.615663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.615671 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:44:21.615778 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.615783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.616549 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.616552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.617950 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.617961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.618376 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.618380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.618474 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.618529 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:44:21.618796 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:44:21.618802 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.618867 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.618943 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1228,"thread_start_us":94,"threads_started":1}
I20260806 00:44:21.625828 9441 raft_consensus.cc:1261] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Refusing update from remote peer 3e340a404fdd4482a28d20d5c34969cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:21.626140 9028 raft_consensus.cc:1261] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Refusing update from remote peer 3e340a404fdd4482a28d20d5c34969cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:21.626453 9612 consensus_queue.cc:1035] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:21.626616 9566 consensus_queue.cc:1035] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:21.663646 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.659954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:58887 (local address 127.29.84.130:46309)
0806 00:44:21.660057 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.660062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.660320 (+ 258us) server_negotiation.cc:407] Connection header received
0806 00:44:21.660442 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.660444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.660503 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.660589 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:21.660823 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.661307 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.662504 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.662725 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.662843 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.663171 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:21.663269 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:21.663353 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.663455 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.663511 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 00:44:21.663668 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.659871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46309 (local address 127.29.84.129:58887)
0806 00:44:21.660222 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.660236 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.660344 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.660611 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.660614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.660621 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:44:21.660723 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.660729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.661422 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.661425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.662386 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.662397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.662857 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.662861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.662950 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.662999 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:44:21.663369 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:44:21.663377 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.663442 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.663526 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":55,"threads_started":1}
I20260806 00:44:21.692181 8801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:21.692341 8801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:21.694541 8801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6: Sending DeleteTablet for 3 replicas of tablet 0ff36909f8104251b1ef620292599149
I20260806 00:44:21.695021 8992 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ff36909f8104251b1ef620292599149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:21 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:41324
I20260806 00:44:21.695439 9191 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ff36909f8104251b1ef620292599149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:21 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:39608
I20260806 00:44:21.695465 9649 tablet_replica.cc:330] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: stopping tablet replica
I20260806 00:44:21.695691 9649 raft_consensus.cc:2229] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:21.695699 9417 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ff36909f8104251b1ef620292599149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:21 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53532
I20260806 00:44:21.695883 9649 raft_consensus.cc:2258] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:21.696642 9649 ts_tablet_manager.cc:1905] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:21.699728 9649 ts_tablet_manager.cc:1918] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:44:21.699949 9651 tablet_replica.cc:330] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: stopping tablet replica
I20260806 00:44:21.700078 9649 log.cc:1196] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/0ff36909f8104251b1ef620292599149
I20260806 00:44:21.700266 9651 raft_consensus.cc:2229] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:21.700451 9650 tablet_replica.cc:330] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: stopping tablet replica
I20260806 00:44:21.700578 9650 raft_consensus.cc:2229] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:44:21.700762 9651 raft_consensus.cc:2258] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:21.700868 9650 raft_consensus.cc:2258] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:21.701040 9649 ts_tablet_manager.cc:1939] T 0ff36909f8104251b1ef620292599149 P 29518182f2e34da2822f61d066619756: Deleting consensus metadata
I20260806 00:44:21.701540 9650 ts_tablet_manager.cc:1905] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:21.702064 9651 ts_tablet_manager.cc:1905] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:21.702443 8783 catalog_manager.cc:4841] TS 29518182f2e34da2822f61d066619756 (127.29.84.129:43893): tablet 0ff36909f8104251b1ef620292599149 (table test-table [id=7003b04736d54554a8deed6d93a06629]) successfully deleted
I20260806 00:44:21.704558 9650 ts_tablet_manager.cc:1918] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:44:21.704660 9650 log.cc:1196] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/0ff36909f8104251b1ef620292599149
I20260806 00:44:21.704941 9650 ts_tablet_manager.cc:1939] T 0ff36909f8104251b1ef620292599149 P 3e340a404fdd4482a28d20d5c34969cb: Deleting consensus metadata
I20260806 00:44:21.705557 8780 catalog_manager.cc:4841] TS 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309): tablet 0ff36909f8104251b1ef620292599149 (table test-table [id=7003b04736d54554a8deed6d93a06629]) successfully deleted
I20260806 00:44:21.707207 9651 ts_tablet_manager.cc:1918] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:44:21.707342 9651 log.cc:1196] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/0ff36909f8104251b1ef620292599149
I20260806 00:44:21.707646 9651 ts_tablet_manager.cc:1939] T 0ff36909f8104251b1ef620292599149 P a9a1e802045d4e5db9c06203bb71b400: Deleting consensus metadata
I20260806 00:44:21.708309 8784 catalog_manager.cc:4841] TS a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169): tablet 0ff36909f8104251b1ef620292599149 (table test-table [id=7003b04736d54554a8deed6d93a06629]) successfully deleted
Aug 06 00:44:21 dist-test-slave-n326 krb5kdc[8738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:44:21 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:21 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:44:21.744405 9622 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:21.738009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45218 (local address 127.29.84.190:44985)
0806 00:44:21.738131 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.738137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.738337 (+ 200us) server_negotiation.cc:407] Connection header received
0806 00:44:21.738491 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.738494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.738547 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.738636 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:21.739419 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.739936 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.740800 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.741046 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.741166 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.744271 (+ 3105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45218: BlockingRecv error: recv got EOF from 127.0.0.1:45218 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 00:44:21 dist-test-slave-n326 krb5kdc[8738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM
I20260806 00:44:21.754083 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.745713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45234 (local address 127.29.84.190:44985)
0806 00:44:21.745818 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.745822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.745906 (+ 84us) server_negotiation.cc:407] Connection header received
0806 00:44:21.745992 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.745995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.746035 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.746119 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:21.746647 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.747463 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.748203 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.748533 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.748644 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.750978 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:21.750981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:21.750983 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:21.751004 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:21.752533 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:21.752990 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:21.752992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:21.752993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:21.753065 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:21.753344 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:21.753346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:21.753347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:21.753499 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:21.753606 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.753846 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.753935 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:44:21.756436 8801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:44:21.756913 8801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:21.761709 8992 tablet_service.cc:1467] Processing CreateTablet for tablet bc90e60c464345eb943e1c0c170dc0eb (DEFAULT_TABLE table=test-table [id=942cb08da18c421bb7dbb2e24781c3b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:21.761955 8992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc90e60c464345eb943e1c0c170dc0eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.761880 9191 tablet_service.cc:1467] Processing CreateTablet for tablet bc90e60c464345eb943e1c0c170dc0eb (DEFAULT_TABLE table=test-table [id=942cb08da18c421bb7dbb2e24781c3b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:21.762147 9191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc90e60c464345eb943e1c0c170dc0eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.763346 9417 tablet_service.cc:1467] Processing CreateTablet for tablet bc90e60c464345eb943e1c0c170dc0eb (DEFAULT_TABLE table=test-table [id=942cb08da18c421bb7dbb2e24781c3b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:21.763624 9417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc90e60c464345eb943e1c0c170dc0eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.765864 9552 tablet_bootstrap.cc:492] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: Bootstrap starting.
I20260806 00:44:21.765941 9551 tablet_bootstrap.cc:492] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: Bootstrap starting.
I20260806 00:44:21.766806 9552 tablet_bootstrap.cc:654] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.766994 9551 tablet_bootstrap.cc:654] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.768038 9551 tablet_bootstrap.cc:492] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: No bootstrap required, opened a new log
I20260806 00:44:21.768105 9552 tablet_bootstrap.cc:492] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: No bootstrap required, opened a new log
I20260806 00:44:21.768113 9551 ts_tablet_manager.cc:1397] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 00:44:21.768179 9552 ts_tablet_manager.cc:1397] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:21.768569 9551 raft_consensus.cc:348] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.768682 9551 raft_consensus.cc:374] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.768718 9551 raft_consensus.cc:729] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29518182f2e34da2822f61d066619756, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.768831 9551 consensus_queue.cc:260] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.768920 9552 raft_consensus.cc:348] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.769032 9552 raft_consensus.cc:374] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.769071 9552 raft_consensus.cc:729] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a1e802045d4e5db9c06203bb71b400, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.769114 9551 ts_tablet_manager.cc:1428] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:44:21.769186 9552 consensus_queue.cc:260] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.769553 9550 tablet_bootstrap.cc:492] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: Bootstrap starting.
I20260806 00:44:21.770529 9550 tablet_bootstrap.cc:654] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.770604 9552 ts_tablet_manager.cc:1428] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:21.772364 9550 tablet_bootstrap.cc:492] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: No bootstrap required, opened a new log
I20260806 00:44:21.772447 9550 ts_tablet_manager.cc:1397] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 00:44:21.772907 9550 raft_consensus.cc:348] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.773022 9550 raft_consensus.cc:374] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.773069 9550 raft_consensus.cc:729] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e340a404fdd4482a28d20d5c34969cb, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.773224 9550 consensus_queue.cc:260] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.773502 9550 ts_tablet_manager.cc:1428] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:44:21.791706 9568 raft_consensus.cc:482] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:21.791822 9568 raft_consensus.cc:504] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.792830 9568 leader_election.cc:288] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29518182f2e34da2822f61d066619756 (127.29.84.129:43893), 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309)
I20260806 00:44:21.797254 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.793315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:52947 (local address 127.29.84.129:43893)
0806 00:44:21.793524 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.793528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.793540 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:44:21.793586 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.793589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.793645 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.793732 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:21.794192 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.794886 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.796269 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.796670 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.796796 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.796845 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.797051 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.797136 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260806 00:44:21.797262 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.793021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:43893 (local address 127.29.84.131:52947)
0806 00:44:21.793355 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.793372 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.793491 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.793838 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.793841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.793859 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:21.794059 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.794066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.795029 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.795032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.796162 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.796169 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.796790 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.796794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.796898 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.796921 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.796995 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.797104 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":27,"thread_start_us":76,"threads_started":1}
I20260806 00:44:21.797353 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.793311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:38043 (local address 127.29.84.130:46309)
0806 00:44:21.793440 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.793445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.793525 (+ 80us) server_negotiation.cc:407] Connection header received
0806 00:44:21.793713 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.793718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.793758 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.793835 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:21.794165 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.794639 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.796136 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.796715 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.796827 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.796871 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.797165 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.797237 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:44:21.797650 9028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc90e60c464345eb943e1c0c170dc0eb" candidate_uuid: "a9a1e802045d4e5db9c06203bb71b400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29518182f2e34da2822f61d066619756" is_pre_election: true
I20260806 00:44:21.797799 9028 raft_consensus.cc:2454] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a1e802045d4e5db9c06203bb71b400 in term 0.
I20260806 00:44:21.798184 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.793070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46309 (local address 127.29.84.131:38043)
0806 00:44:21.793445 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:21.793458 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:21.793586 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:21.793847 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:21.793849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:21.793866 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:21.794051 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.794058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.794744 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.794750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.796002 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:21.796012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.796913 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.796916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:21.797013 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.797051 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 00:44:21.797949 (+ 898us) client_negotiation.cc:240] Negotiation successful
0806 00:44:21.798027 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":32,"thread_start_us":59,"threads_started":1}
I20260806 00:44:21.798127 9336 leader_election.cc:302] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29518182f2e34da2822f61d066619756, a9a1e802045d4e5db9c06203bb71b400; no voters:
I20260806 00:44:21.798381 9568 raft_consensus.cc:2781] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:21.798442 9568 raft_consensus.cc:482] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:21.798487 9568 raft_consensus.cc:3037] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.798640 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc90e60c464345eb943e1c0c170dc0eb" candidate_uuid: "a9a1e802045d4e5db9c06203bb71b400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e340a404fdd4482a28d20d5c34969cb" is_pre_election: true
I20260806 00:44:21.798776 9213 raft_consensus.cc:2454] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a1e802045d4e5db9c06203bb71b400 in term 0.
I20260806 00:44:21.799616 9568 raft_consensus.cc:504] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.799949 9568 leader_election.cc:288] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [CANDIDATE]: Term 1 election: Requested vote from peers 29518182f2e34da2822f61d066619756 (127.29.84.129:43893), 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309)
I20260806 00:44:21.800266 9028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc90e60c464345eb943e1c0c170dc0eb" candidate_uuid: "a9a1e802045d4e5db9c06203bb71b400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29518182f2e34da2822f61d066619756"
I20260806 00:44:21.800285 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc90e60c464345eb943e1c0c170dc0eb" candidate_uuid: "a9a1e802045d4e5db9c06203bb71b400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e340a404fdd4482a28d20d5c34969cb"
I20260806 00:44:21.800371 9028 raft_consensus.cc:3037] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.800412 9213 raft_consensus.cc:3037] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:21.801201 9213 raft_consensus.cc:2454] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a1e802045d4e5db9c06203bb71b400 in term 1.
I20260806 00:44:21.801208 9028 raft_consensus.cc:2454] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a1e802045d4e5db9c06203bb71b400 in term 1.
I20260806 00:44:21.801469 9336 leader_election.cc:302] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29518182f2e34da2822f61d066619756, a9a1e802045d4e5db9c06203bb71b400; no voters:
I20260806 00:44:21.801640 9568 raft_consensus.cc:2781] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:21.801862 9568 raft_consensus.cc:686] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 1 LEADER]: Becoming Leader. State: Replica: a9a1e802045d4e5db9c06203bb71b400, State: Running, Role: LEADER
I20260806 00:44:21.802124 9568 consensus_queue.cc:237] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.804240 8801 catalog_manager.cc:5495] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131). New cstate: current_term: 1 leader_uuid: "a9a1e802045d4e5db9c06203bb71b400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } health_report { overall_health: HEALTHY } } }
I20260806 00:44:21.813884 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:21.810583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53544 (local address 127.29.84.131:45169)
0806 00:44:21.810683 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:21.810689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:21.810707 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:44:21.810885 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:21.810888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:21.810945 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:21.811050 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:21.811289 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.811749 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.812570 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:21.812804 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:21.812910 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:21.813165 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:21.813547 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:21.813666 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:21.813726 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 00:44:21.813773 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:44:21.816421 9028 raft_consensus.cc:1261] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Refusing update from remote peer a9a1e802045d4e5db9c06203bb71b400: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:21.816421 9213 raft_consensus.cc:1261] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Refusing update from remote peer a9a1e802045d4e5db9c06203bb71b400: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:21.816932 9568 consensus_queue.cc:1035] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:21.817138 9680 consensus_queue.cc:1035] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:21.828100 8800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:21.828250 8800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:21.830255 8800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6: Sending DeleteTablet for 3 replicas of tablet bc90e60c464345eb943e1c0c170dc0eb
I20260806 00:44:21.830727 8992 tablet_service.cc:1514] Processing DeleteTablet for tablet bc90e60c464345eb943e1c0c170dc0eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:21 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:41324
I20260806 00:44:21.830893 9649 tablet_replica.cc:330] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: stopping tablet replica
I20260806 00:44:21.831010 9649 raft_consensus.cc:2229] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:21.831157 9649 raft_consensus.cc:2258] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:21.831534 9191 tablet_service.cc:1514] Processing DeleteTablet for tablet bc90e60c464345eb943e1c0c170dc0eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:21 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:39608
I20260806 00:44:21.831609 9417 tablet_service.cc:1514] Processing DeleteTablet for tablet bc90e60c464345eb943e1c0c170dc0eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:21 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53532
I20260806 00:44:21.831732 9650 tablet_replica.cc:330] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: stopping tablet replica
I20260806 00:44:21.831858 9651 tablet_replica.cc:330] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: stopping tablet replica
I20260806 00:44:21.831863 9650 raft_consensus.cc:2229] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:21.831856 8800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 00:44:21.831962 9651 raft_consensus.cc:2229] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:44:21.831995 9650 raft_consensus.cc:2258] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:21.832203 9651 raft_consensus.cc:2258] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 00:44:21.832227 8800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:21.832803 9650 ts_tablet_manager.cc:1905] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:21.833302 9649 ts_tablet_manager.cc:1905] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:21.835243 9649 ts_tablet_manager.cc:1918] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:21.835332 9649 log.cc:1196] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/bc90e60c464345eb943e1c0c170dc0eb
I20260806 00:44:21.835448 9650 ts_tablet_manager.cc:1918] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:21.835532 9650 log.cc:1196] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/bc90e60c464345eb943e1c0c170dc0eb
I20260806 00:44:21.835628 9649 ts_tablet_manager.cc:1939] T bc90e60c464345eb943e1c0c170dc0eb P 29518182f2e34da2822f61d066619756: Deleting consensus metadata
I20260806 00:44:21.835865 9650 ts_tablet_manager.cc:1939] T bc90e60c464345eb943e1c0c170dc0eb P 3e340a404fdd4482a28d20d5c34969cb: Deleting consensus metadata
I20260806 00:44:21.836308 8783 catalog_manager.cc:4841] TS 29518182f2e34da2822f61d066619756 (127.29.84.129:43893): tablet bc90e60c464345eb943e1c0c170dc0eb (table test-table [id=942cb08da18c421bb7dbb2e24781c3b6]) successfully deleted
I20260806 00:44:21.836534 9651 ts_tablet_manager.cc:1905] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:21.838943 9651 ts_tablet_manager.cc:1918] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:21.839030 9651 log.cc:1196] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/bc90e60c464345eb943e1c0c170dc0eb
I20260806 00:44:21.839337 9651 ts_tablet_manager.cc:1939] T bc90e60c464345eb943e1c0c170dc0eb P a9a1e802045d4e5db9c06203bb71b400: Deleting consensus metadata
I20260806 00:44:21.840066 8784 catalog_manager.cc:4841] TS a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169): tablet bc90e60c464345eb943e1c0c170dc0eb (table test-table [id=942cb08da18c421bb7dbb2e24781c3b6]) successfully deleted
I20260806 00:44:21.841117 8780 catalog_manager.cc:4841] TS 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309): tablet bc90e60c464345eb943e1c0c170dc0eb (table test-table [id=942cb08da18c421bb7dbb2e24781c3b6]) successfully deleted
I20260806 00:44:21.842501 8992 tablet_service.cc:1467] Processing CreateTablet for tablet f4829bebfae445ee8c243d4ea08a94b7 (DEFAULT_TABLE table=test-table [id=c51ed929858947b0b92d0453177d7284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:21.842774 8992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4829bebfae445ee8c243d4ea08a94b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.842871 9191 tablet_service.cc:1467] Processing CreateTablet for tablet f4829bebfae445ee8c243d4ea08a94b7 (DEFAULT_TABLE table=test-table [id=c51ed929858947b0b92d0453177d7284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:21.843164 9191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4829bebfae445ee8c243d4ea08a94b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.845113 9551 tablet_bootstrap.cc:492] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756: Bootstrap starting.
I20260806 00:44:21.845294 9550 tablet_bootstrap.cc:492] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: Bootstrap starting.
I20260806 00:44:21.845912 9551 tablet_bootstrap.cc:654] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.846206 9550 tablet_bootstrap.cc:654] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.847146 9417 tablet_service.cc:1467] Processing CreateTablet for tablet f4829bebfae445ee8c243d4ea08a94b7 (DEFAULT_TABLE table=test-table [id=c51ed929858947b0b92d0453177d7284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:21.847386 9417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4829bebfae445ee8c243d4ea08a94b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:21.847725 9551 tablet_bootstrap.cc:492] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756: No bootstrap required, opened a new log
I20260806 00:44:21.847795 9551 ts_tablet_manager.cc:1397] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 00:44:21.848133 9551 raft_consensus.cc:348] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.848266 9551 raft_consensus.cc:374] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.848299 9551 raft_consensus.cc:729] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29518182f2e34da2822f61d066619756, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.848421 9551 consensus_queue.cc:260] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.848697 9551 ts_tablet_manager.cc:1428] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:44:21.849522 9552 tablet_bootstrap.cc:492] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: Bootstrap starting.
I20260806 00:44:21.850149 9552 tablet_bootstrap.cc:654] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:21.851768 9550 tablet_bootstrap.cc:492] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: No bootstrap required, opened a new log
I20260806 00:44:21.851838 9550 ts_tablet_manager.cc:1397] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260806 00:44:21.852218 9550 raft_consensus.cc:348] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.852314 9550 raft_consensus.cc:374] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.852342 9550 raft_consensus.cc:729] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e340a404fdd4482a28d20d5c34969cb, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.852443 9550 consensus_queue.cc:260] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.852676 9550 ts_tablet_manager.cc:1428] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:44:21.855921 9552 tablet_bootstrap.cc:492] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: No bootstrap required, opened a new log
I20260806 00:44:21.855991 9552 ts_tablet_manager.cc:1397] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260806 00:44:21.856462 9552 raft_consensus.cc:348] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.856559 9552 raft_consensus.cc:374] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:21.856592 9552 raft_consensus.cc:729] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a1e802045d4e5db9c06203bb71b400, State: Initialized, Role: FOLLOWER
I20260806 00:44:21.856719 9552 consensus_queue.cc:260] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:21.856980 9552 ts_tablet_manager.cc:1428] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:44:22.291929 9680 raft_consensus.cc:482] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:22.292073 9680 raft_consensus.cc:504] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:22.292515 9680 leader_election.cc:288] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29518182f2e34da2822f61d066619756 (127.29.84.129:43893), 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309)
I20260806 00:44:22.292840 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4829bebfae445ee8c243d4ea08a94b7" candidate_uuid: "a9a1e802045d4e5db9c06203bb71b400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e340a404fdd4482a28d20d5c34969cb" is_pre_election: true
I20260806 00:44:22.292858 9028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4829bebfae445ee8c243d4ea08a94b7" candidate_uuid: "a9a1e802045d4e5db9c06203bb71b400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29518182f2e34da2822f61d066619756" is_pre_election: true
I20260806 00:44:22.293012 9213 raft_consensus.cc:2454] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a1e802045d4e5db9c06203bb71b400 in term 0.
I20260806 00:44:22.293021 9028 raft_consensus.cc:2454] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a1e802045d4e5db9c06203bb71b400 in term 0.
I20260806 00:44:22.293313 9336 leader_election.cc:302] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29518182f2e34da2822f61d066619756, a9a1e802045d4e5db9c06203bb71b400; no voters:
I20260806 00:44:22.293500 9680 raft_consensus.cc:2781] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:22.293563 9680 raft_consensus.cc:482] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:22.293591 9680 raft_consensus.cc:3037] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:22.294582 9680 raft_consensus.cc:504] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:22.294911 9680 leader_election.cc:288] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [CANDIDATE]: Term 1 election: Requested vote from peers 29518182f2e34da2822f61d066619756 (127.29.84.129:43893), 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309)
I20260806 00:44:22.295220 9028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4829bebfae445ee8c243d4ea08a94b7" candidate_uuid: "a9a1e802045d4e5db9c06203bb71b400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29518182f2e34da2822f61d066619756"
I20260806 00:44:22.295351 9028 raft_consensus.cc:3037] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:22.295315 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4829bebfae445ee8c243d4ea08a94b7" candidate_uuid: "a9a1e802045d4e5db9c06203bb71b400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e340a404fdd4482a28d20d5c34969cb"
I20260806 00:44:22.295459 9213 raft_consensus.cc:3037] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:22.296295 9028 raft_consensus.cc:2454] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a1e802045d4e5db9c06203bb71b400 in term 1.
I20260806 00:44:22.296332 9213 raft_consensus.cc:2454] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a1e802045d4e5db9c06203bb71b400 in term 1.
I20260806 00:44:22.296564 9336 leader_election.cc:302] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29518182f2e34da2822f61d066619756, a9a1e802045d4e5db9c06203bb71b400; no voters:
I20260806 00:44:22.296738 9680 raft_consensus.cc:2781] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:22.296857 9680 raft_consensus.cc:686] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 1 LEADER]: Becoming Leader. State: Replica: a9a1e802045d4e5db9c06203bb71b400, State: Running, Role: LEADER
I20260806 00:44:22.297014 9680 consensus_queue.cc:237] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } }
I20260806 00:44:22.298763 8800 catalog_manager.cc:5495] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131). New cstate: current_term: 1 leader_uuid: "a9a1e802045d4e5db9c06203bb71b400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a1e802045d4e5db9c06203bb71b400" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45169 } health_report { overall_health: HEALTHY } } }
I20260806 00:44:22.427524 9714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:22.424086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:45369 (local address 127.29.84.129:43893)
0806 00:44:22.424328 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:22.424341 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:22.424371 (+ 30us) server_negotiation.cc:407] Connection header received
0806 00:44:22.424562 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:22.424565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:22.424616 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:22.424697 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:22.424974 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:22.425565 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:22.426433 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:22.426658 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:22.426764 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:22.427049 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:22.427139 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:22.427220 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:22.427316 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 00:44:22.427361 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
I20260806 00:44:22.427529 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:22.423946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:43893 (local address 127.29.84.131:45369)
0806 00:44:22.424305 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:22.424321 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:22.424456 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:22.424721 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:22.424724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:22.424733 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:44:22.424834 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:22.424851 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:22.425673 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:22.425679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:22.426305 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:22.426314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:22.426769 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:22.426772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:22.426859 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:22.426911 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:44:22.427228 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:44:22.427237 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:44:22.427297 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 00:44:22.427382 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
I20260806 00:44:22.433218 9213 raft_consensus.cc:1261] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Refusing update from remote peer a9a1e802045d4e5db9c06203bb71b400: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:22.433386 9028 raft_consensus.cc:1261] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Refusing update from remote peer a9a1e802045d4e5db9c06203bb71b400: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:22.433648 9568 consensus_queue.cc:1035] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e340a404fdd4482a28d20d5c34969cb" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:22.433804 9680 consensus_queue.cc:1035] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29518182f2e34da2822f61d066619756" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 43893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:22.449360 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:22.446136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:33487 (local address 127.29.84.131:45169)
0806 00:44:22.446426 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:22.446438 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:22.446557 (+ 119us) server_negotiation.cc:407] Connection header received
0806 00:44:22.446605 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:22.446607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:22.446643 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:22.446706 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:22.446846 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:22.447387 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:22.448197 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:22.448483 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:22.448592 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:22.448870 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:22.448959 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:22.449035 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:22.449162 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 00:44:22.449215 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":121,"threads_started":1}
I20260806 00:44:22.449384 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:22.446058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45169 (local address 127.29.84.129:33487)
0806 00:44:22.446447 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:22.446461 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:22.446543 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:22.446710 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:22.446711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:22.446716 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:44:22.446790 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:22.446793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:22.447487 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:22.447489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:22.448068 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:22.448076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:22.448608 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:22.448612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:22.448707 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:22.448736 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:44:22.449060 (+ 324us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:44:22.449066 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:44:22.449119 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 00:44:22.449228 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":107,"threads_started":1}
I20260806 00:44:22.472559 8800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:22.472698 8800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:22.474501 8800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb3054a78d1c4c27950d41e1602bacb6: Sending DeleteTablet for 3 replicas of tablet f4829bebfae445ee8c243d4ea08a94b7
I20260806 00:44:22.474965 8992 tablet_service.cc:1514] Processing DeleteTablet for tablet f4829bebfae445ee8c243d4ea08a94b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:22 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:41324
I20260806 00:44:22.475077 9191 tablet_service.cc:1514] Processing DeleteTablet for tablet f4829bebfae445ee8c243d4ea08a94b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:22 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:39608
I20260806 00:44:22.475334 9729 tablet_replica.cc:330] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756: stopping tablet replica
I20260806 00:44:22.475454 9417 tablet_service.cc:1514] Processing DeleteTablet for tablet f4829bebfae445ee8c243d4ea08a94b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:22 UTC) from {username='oryx', principal='oryx/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53532
I20260806 00:44:22.475472 9729 raft_consensus.cc:2229] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:22.475574 9729 raft_consensus.cc:2258] T f4829bebfae445ee8c243d4ea08a94b7 P 29518182f2e34da2822f61d066619756 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:22.475795 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 8905
I20260806 00:44:22.475837 9731 tablet_replica.cc:330] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: stopping tablet replica
I20260806 00:44:22.475965 9731 raft_consensus.cc:2229] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:44:22.476116 9731 raft_consensus.cc:2258] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:22.476676 9731 ts_tablet_manager.cc:1905] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:22.478590 9731 ts_tablet_manager.cc:1918] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:44:22.478643 9731 log.cc:1196] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/f4829bebfae445ee8c243d4ea08a94b7
I20260806 00:44:22.478859 9731 ts_tablet_manager.cc:1939] T f4829bebfae445ee8c243d4ea08a94b7 P a9a1e802045d4e5db9c06203bb71b400: Deleting consensus metadata
I20260806 00:44:22.480507 8784 catalog_manager.cc:4841] TS a9a1e802045d4e5db9c06203bb71b400 (127.29.84.131:45169): tablet f4829bebfae445ee8c243d4ea08a94b7 (table test-table [id=c51ed929858947b0b92d0453177d7284]) successfully deleted
I20260806 00:44:22.483232 9730 tablet_replica.cc:330] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: stopping tablet replica
I20260806 00:44:22.483374 9730 raft_consensus.cc:2229] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:22.483489 9730 raft_consensus.cc:2258] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:22.483999 9730 ts_tablet_manager.cc:1905] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:22.485877 9730 ts_tablet_manager.cc:1918] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:44:22.485983 9730 log.cc:1196] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/f4829bebfae445ee8c243d4ea08a94b7
I20260806 00:44:22.486232 9730 ts_tablet_manager.cc:1939] T f4829bebfae445ee8c243d4ea08a94b7 P 3e340a404fdd4482a28d20d5c34969cb: Deleting consensus metadata
I20260806 00:44:22.486783 8780 catalog_manager.cc:4841] TS 3e340a404fdd4482a28d20d5c34969cb (127.29.84.130:46309): tablet f4829bebfae445ee8c243d4ea08a94b7 (table test-table [id=c51ed929858947b0b92d0453177d7284]) successfully deleted
W20260806 00:44:22.490164 8783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:43893 (error 108)
I20260806 00:44:22.490942 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 9106
I20260806 00:44:22.496829 9736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:22.490766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:43893 (local address 127.0.0.1:41354)
0806 00:44:22.490996 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:22.496746 (+ 5750us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
W20260806 00:44:22.496974 8783 catalog_manager.cc:4584] TS 29518182f2e34da2822f61d066619756 (127.29.84.129:43893): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4829bebfae445ee8c243d4ea08a94b7: Network error: Client connection negotiation failed: client connection to 127.29.84.129:43893: connect: Connection refused (error 111)
I20260806 00:44:22.502897 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 9311
I20260806 00:44:22.512245 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 8760
2026-08-06T00:44:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3870 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:22 dist-test-slave-n326 krb5kdc[9741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:22 dist-test-slave-n326 krb5kdc[9741](info): set up 2 sockets
Aug 06 00:44:22 dist-test-slave-n326 krb5kdc[9741](info): commencing operation
krb5kdc: starting...
W20260806 00:44:23.727778 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:23 dist-test-slave-n326 krb5kdc[9741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:44:23.843811 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:36715
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:36715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:23.947705 9761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:23.947906 9761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:23.947942 9761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:23.951777 9761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:23.951822 9761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:23.951841 9761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:23.951857 9761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:23.951912 9761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:23.956749 9761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:36715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:36715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:23.957651 9761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:23.958549 9761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:23.966729 9775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:23.966748 9771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:23.966770 9772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:23.967020 9761 server_base.cc:1034] running on GCE node
I20260806 00:44:23.967684 9761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:23.968765 9761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:23.969980 9761 hybrid_clock.cc:648] HybridClock initialized: now 1785977063969963 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T00:44:23Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1462 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:24 dist-test-slave-n326 krb5kdc[9785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:24 dist-test-slave-n326 krb5kdc[9785](info): set up 2 sockets
Aug 06 00:44:24 dist-test-slave-n326 krb5kdc[9785](info): commencing operation
krb5kdc: starting...
W20260806 00:44:25.181174 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:25 dist-test-slave-n326 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:44:25.298606 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:33257
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:33257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:25.405429 9802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:25.405670 9802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:25.405747 9802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:25.409746 9802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:25.409824 9802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:25.409862 9802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:25.409921 9802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:25.409962 9802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:25.414991 9802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:33257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:33257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:25.415872 9802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:25.416759 9802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:25.424817 9810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:25.424825 9813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:25.424835 9811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:25.425091 9802 server_base.cc:1034] running on GCE node
I20260806 00:44:25.425730 9802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:25.426731 9802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:25.427928 9802 hybrid_clock.cc:648] HybridClock initialized: now 1785977065427909 us; error 34 us; skew 500 ppm
Aug 06 00:44:25 dist-test-slave-n326 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:25.430660 9802 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:44:25.431576 9802 webserver.cc:460] Webserver started at http://127.29.84.190:41623/ using document root <none> and password file <none>
I20260806 00:44:25.432035 9802 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:25.432091 9802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:25.432242 9802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:25.434032 9802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "fb59852e9ab74e4b93373197f9661960"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "1ee450c3d9856f0830912034cdae0c02"
server_key_iv: "c91169fddd13d83c36ee2e237bb8fdc6"
server_key_version: "encryptionkey@0"
I20260806 00:44:25.434510 9802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "fb59852e9ab74e4b93373197f9661960"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "1ee450c3d9856f0830912034cdae0c02"
server_key_iv: "c91169fddd13d83c36ee2e237bb8fdc6"
server_key_version: "encryptionkey@0"
I20260806 00:44:25.437904 9802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:44:25.440157 9821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:25.441248 9802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:25.441390 9802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "fb59852e9ab74e4b93373197f9661960"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "1ee450c3d9856f0830912034cdae0c02"
server_key_iv: "c91169fddd13d83c36ee2e237bb8fdc6"
server_key_version: "encryptionkey@0"
I20260806 00:44:25.441509 9802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:25.453423 9802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:25.456830 9802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:25.457013 9802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:25.463874 9802 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:33257
I20260806 00:44:25.463886 9899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:33257 every 8 connection(s)
I20260806 00:44:25.464710 9802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:25.467231 9900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:25.472200 9900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: Bootstrap starting.
I20260806 00:44:25.473986 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 9802
I20260806 00:44:25.474148 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:25.474411 30034 external_mini_cluster.cc:1428] Setting key 34ce7ae9f3af45221abb0a1ee7842628
I20260806 00:44:25.474421 9900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:25.475234 9900 log.cc:824] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:25.477291 9900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: No bootstrap required, opened a new log
I20260806 00:44:25.481449 9900 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } }
I20260806 00:44:25.481680 9900 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:25.481801 9900 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb59852e9ab74e4b93373197f9661960, State: Initialized, Role: FOLLOWER
I20260806 00:44:25.482206 9900 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } }
I20260806 00:44:25.482354 9900 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:25.482445 9900 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:25.482563 9900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 00:44:25 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:25.483462 9900 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } }
I20260806 00:44:25.483780 9900 leader_election.cc:302] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb59852e9ab74e4b93373197f9661960; no voters:
I20260806 00:44:25.484114 9900 leader_election.cc:288] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:25.484267 9910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:25.484819 9910 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 1 LEADER]: Becoming Leader. State: Replica: fb59852e9ab74e4b93373197f9661960, State: Running, Role: LEADER
I20260806 00:44:25.485266 9900 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:25.485397 9910 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } }
I20260806 00:44:25.487191 9916 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb59852e9ab74e4b93373197f9661960. Latest consensus state: current_term: 1 leader_uuid: "fb59852e9ab74e4b93373197f9661960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } } }
I20260806 00:44:25.487354 9916 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:25.487018 9912 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb59852e9ab74e4b93373197f9661960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } } }
I20260806 00:44:25.487635 9912 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:25.489316 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:25.475543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.29.84.190:33257)
0806 00:44:25.475919 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:25.475933 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:25.475988 (+ 55us) server_negotiation.cc:407] Connection header received
0806 00:44:25.476768 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:25.476789 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:25.477144 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:25.477477 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:25.478256 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.479772 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.480575 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.481238 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.481407 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:25.484061 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:25.484069 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:25.484087 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:25.484115 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:25.486190 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:25.486730 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:25.486736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:25.486742 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:25.486902 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:25.487308 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:25.487311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:25.487312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:25.487852 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:25.488072 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:25.488331 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 00:44:25.488643 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":103,"threads_started":1}
W20260806 00:44:25.489948 9937 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:44:25.490020 9937 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:44:25.490254 9938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:25.491031 9938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:25.495908 9938 catalog_manager.cc:1347] Generated new cluster ID: 5d0eeb052b884802a510554d6e69c7a0
I20260806 00:44:25.495967 9938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:25.502023 9938 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:25.502941 9938 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:25.517175 9938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: Generated new TSK 0
I20260806 00:44:25.517741 9938 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:44:25.550251 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33257
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:25.662340 9945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:25.662559 9945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:25.662606 9945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:25.666626 9945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:25.666674 9945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:25.666759 9945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:25.671700 9945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:25.672639 9945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:25.673568 9945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:44:25.682796 9945 server_base.cc:1034] running on GCE node
W20260806 00:44:25.682838 9954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:25.682945 9957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:25.682801 9955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:25.683490 9945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:25.684077 9945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:25.685242 9945 hybrid_clock.cc:648] HybridClock initialized: now 1785977065685216 us; error 39 us; skew 500 ppm
Aug 06 00:44:25 dist-test-slave-n326 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:25.688391 9945 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:44:25.689203 9945 webserver.cc:460] Webserver started at http://127.29.84.129:46429/ using document root <none> and password file <none>
I20260806 00:44:25.689682 9945 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:25.689738 9945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:25.689886 9945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:25.691783 9945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "7eaec7cd60224759a11910bae70793f2"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "2f315969e0dd9b69f8ebb9bceda612e6"
server_key_iv: "e5398efa6842bdce90e015e801b05cb0"
server_key_version: "encryptionkey@0"
I20260806 00:44:25.692265 9945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "7eaec7cd60224759a11910bae70793f2"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "2f315969e0dd9b69f8ebb9bceda612e6"
server_key_iv: "e5398efa6842bdce90e015e801b05cb0"
server_key_version: "encryptionkey@0"
I20260806 00:44:25.696262 9945 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:44:25.698498 9966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:25.699705 9945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:25.699820 9945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "7eaec7cd60224759a11910bae70793f2"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "2f315969e0dd9b69f8ebb9bceda612e6"
server_key_iv: "e5398efa6842bdce90e015e801b05cb0"
server_key_version: "encryptionkey@0"
I20260806 00:44:25.699923 9945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:25.724015 9945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:25.727159 9945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:25.727351 9945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:25.727934 9945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:25.728716 9945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:25.728762 9945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:25.728830 9945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:25.728863 9945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:25.737736 9945 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:42053
I20260806 00:44:25.737774 10139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:42053 every 8 connection(s)
I20260806 00:44:25.738579 9945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:25.746605 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 9945
I20260806 00:44:25.746786 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:25.747205 30034 external_mini_cluster.cc:1428] Setting key 051b7343caf7b143d2c19396c78c38cc
Aug 06 00:44:25 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:25.752461 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:25.740262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:34869 (local address 127.29.84.190:33257)
0806 00:44:25.740414 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:25.740420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:25.741532 (+ 1112us) server_negotiation.cc:407] Connection header received
0806 00:44:25.742528 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:25.742531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:25.742582 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:25.742671 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:25.743837 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.744568 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.745747 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.745989 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.746079 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:25.748811 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:25.748815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:25.748817 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:25.748839 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:25.750291 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:25.750894 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:25.750898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:25.750899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:25.750986 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:25.751400 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:25.751403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:25.751405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:25.751582 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:25.751699 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:25.752173 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 00:44:25.752294 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:44:25.753301 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:25.740651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:34869)
0806 00:44:25.741362 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:25.741426 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:25.742185 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:25.742826 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:25.742832 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:25.743250 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:25.743701 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:25.743714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.744685 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.744687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:25.745640 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:25.745651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.746125 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.746128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:25.746256 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:25.746912 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:25.746933 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:25.748653 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:25.750412 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:25.750429 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:25.750447 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:25.750768 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:25.751102 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:25.751114 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:25.751116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:25.751287 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:25.751697 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:25.751704 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:25.751934 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 00:44:25.752113 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 00:44:25.752411 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":137,"threads_started":1}
I20260806 00:44:25.754458 10140 heartbeater.cc:347] Connected to a master server at 127.29.84.190:33257
I20260806 00:44:25.754699 10140 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:25.755214 10140 heartbeater.cc:511] Master 127.29.84.190:33257 requested a full tablet report, sending...
I20260806 00:44:25.756837 9849 ts_manager.cc:194] Registered new tserver with Master: 7eaec7cd60224759a11910bae70793f2 (127.29.84.129:42053)
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:44:25.758126 9849 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:34869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:44:25.774756 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33257
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:25.882500 10147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:25.882726 10147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:25.882766 10147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:25.886924 10147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:25.886976 10147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:25.887030 10147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:25.892118 10147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:25.892920 10147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:25.893793 10147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:25.902545 10154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:25.902554 10157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:25.902659 10153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:25.902832 10147 server_base.cc:1034] running on GCE node
I20260806 00:44:25.903296 10147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:25.903914 10147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:25.905072 10147 hybrid_clock.cc:648] HybridClock initialized: now 1785977065905045 us; error 37 us; skew 500 ppm
Aug 06 00:44:25 dist-test-slave-n326 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:25.907758 10147 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:44:25.908579 10147 webserver.cc:460] Webserver started at http://127.29.84.130:41369/ using document root <none> and password file <none>
I20260806 00:44:25.909142 10147 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:25.909237 10147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:25.909458 10147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:25.911412 10147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "cbe89b41da2b42589722abb70313bf61"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "3359a0efac963d8ddd44043869977b28"
server_key_iv: "3e46386486f3bd6bdd2726230f8b51de"
server_key_version: "encryptionkey@0"
I20260806 00:44:25.911960 10147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "cbe89b41da2b42589722abb70313bf61"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "3359a0efac963d8ddd44043869977b28"
server_key_iv: "3e46386486f3bd6bdd2726230f8b51de"
server_key_version: "encryptionkey@0"
I20260806 00:44:25.915417 10147 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:44:25.917652 10168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:25.918793 10147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:25.918934 10147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "cbe89b41da2b42589722abb70313bf61"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "3359a0efac963d8ddd44043869977b28"
server_key_iv: "3e46386486f3bd6bdd2726230f8b51de"
server_key_version: "encryptionkey@0"
I20260806 00:44:25.919039 10147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:25.940223 10147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:25.943392 10147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:25.943585 10147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:25.944145 10147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:25.945030 10147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:25.945108 10147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:25.945202 10147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:25.945241 10147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:25.954226 10147 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:34223
I20260806 00:44:25.954274 10335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:34223 every 8 connection(s)
I20260806 00:44:25.955083 10147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:25.960399 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 10147
I20260806 00:44:25.960539 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:25.960825 30034 external_mini_cluster.cc:1428] Setting key 19738ac586bc17a7f76e2e1243bd5102
Aug 06 00:44:25 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:25.968394 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:25.956765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:59133 (local address 127.29.84.190:33257)
0806 00:44:25.956919 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:25.956923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:25.957851 (+ 928us) server_negotiation.cc:407] Connection header received
0806 00:44:25.958745 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:25.958748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:25.958837 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:25.958934 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:25.960076 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.960690 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.961876 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.962107 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.962227 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:25.964640 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:25.964646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:25.964650 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:25.964682 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:25.966165 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:25.966765 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:25.966767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:25.966769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:25.966838 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:25.967280 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:25.967282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:25.967283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:25.967503 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:25.967583 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:25.968189 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 00:44:25.968283 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:44:25.969571 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:25.957089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:59133)
0806 00:44:25.957705 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:25.957744 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:25.958545 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:25.959153 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:25.959160 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:25.959486 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:25.959924 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:25.959937 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.960819 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.960823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:25.961743 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:25.961753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:25.962261 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:25.962266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:25.962435 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:25.962998 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:25.963014 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:25.964455 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:25.966283 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:25.966292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:25.966307 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:25.966664 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:25.966938 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:25.966944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:25.966949 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:25.967185 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:25.967623 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:25.967630 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:25.967930 (+ 300us) client_negotiation.cc:769] Sending connection context
0806 00:44:25.968136 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 00:44:25.968499 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":101,"threads_started":1}
I20260806 00:44:25.970741 10336 heartbeater.cc:347] Connected to a master server at 127.29.84.190:33257
I20260806 00:44:25.970973 10336 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:25.971567 10336 heartbeater.cc:511] Master 127.29.84.190:33257 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:44:25.972616 9849 ts_manager.cc:194] Registered new tserver with Master: cbe89b41da2b42589722abb70313bf61 (127.29.84.130:34223)
I20260806 00:44:25.973317 9849 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:59133
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:44:25.989962 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33257
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:26.097560 10343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:26.097770 10343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:26.097810 10343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:26.101924 10343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:26.101975 10343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:26.102072 10343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:26.107165 10343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:26.107980 10343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:26.108908 10343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:44:26.117892 10343 server_base.cc:1034] running on GCE node
W20260806 00:44:26.117851 10354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:26.117972 10349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:26.117857 10350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:26.118474 10343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:26.119050 10343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:26.120303 10343 hybrid_clock.cc:648] HybridClock initialized: now 1785977066120296 us; error 28 us; skew 500 ppm
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977066, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:26.123018 10343 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:44:26.123826 10343 webserver.cc:460] Webserver started at http://127.29.84.131:40969/ using document root <none> and password file <none>
I20260806 00:44:26.124326 10343 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:26.124416 10343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:26.124635 10343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:26.126490 10343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "261cbe1dadd04e3e97cba0e6cf49c019"
format_stamp: "Formatted at 2026-08-06 00:44:26 on dist-test-slave-n326"
server_key: "43304d7052167ddc3528b220a45a9b1c"
server_key_iv: "ef54b784ff221d2ddde02a242fd93b0a"
server_key_version: "encryptionkey@0"
I20260806 00:44:26.127018 10343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "261cbe1dadd04e3e97cba0e6cf49c019"
format_stamp: "Formatted at 2026-08-06 00:44:26 on dist-test-slave-n326"
server_key: "43304d7052167ddc3528b220a45a9b1c"
server_key_iv: "ef54b784ff221d2ddde02a242fd93b0a"
server_key_version: "encryptionkey@0"
I20260806 00:44:26.130354 10343 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260806 00:44:26.132529 10365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.133623 10343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 00:44:26.133754 10343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "261cbe1dadd04e3e97cba0e6cf49c019"
format_stamp: "Formatted at 2026-08-06 00:44:26 on dist-test-slave-n326"
server_key: "43304d7052167ddc3528b220a45a9b1c"
server_key_iv: "ef54b784ff221d2ddde02a242fd93b0a"
server_key_version: "encryptionkey@0"
I20260806 00:44:26.133854 10343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:26.151868 10343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:26.155570 10343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:26.155772 10343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:26.156306 10343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:26.157070 10343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:26.157146 10343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.157209 10343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:26.157239 10343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.165549 10343 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:38773
I20260806 00:44:26.165562 10528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:38773 every 8 connection(s)
I20260806 00:44:26.166319 10343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977066, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:26.175895 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 10343
I20260806 00:44:26.176049 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:26.176409 30034 external_mini_cluster.cc:1428] Setting key 691a675a783c57f61f02980a8e70b136
I20260806 00:44:26.179250 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:26.167947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:33185 (local address 127.29.84.190:33257)
0806 00:44:26.168103 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:26.168108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:26.168854 (+ 746us) server_negotiation.cc:407] Connection header received
0806 00:44:26.169998 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:26.170001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:26.170049 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:26.170120 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:26.171182 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.171655 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.172729 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.172936 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.173012 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.175244 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:26.175248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:26.175250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:26.175273 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:26.176909 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:26.177434 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:26.177438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:26.177440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:26.177547 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:26.177922 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:26.177926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:26.177927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:26.178128 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:26.178204 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:26.178981 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 00:44:26.179124 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:44:26.179868 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:26.168273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:33185)
0806 00:44:26.168712 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:26.168752 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:26.169807 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:26.170268 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:26.170274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:26.170576 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:26.170993 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.171005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.171752 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.171754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.172620 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.172629 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.173055 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.173058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.173187 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.173661 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:26.173676 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:26.175028 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:26.177027 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:26.177035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:26.177060 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:26.177313 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:26.177641 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:26.177643 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:26.177645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:26.177829 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:26.178233 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:26.178239 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:26.178503 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 00:44:26.178707 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 00:44:26.178990 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":107,"threads_started":1}
I20260806 00:44:26.180997 10529 heartbeater.cc:347] Connected to a master server at 127.29.84.190:33257
I20260806 00:44:26.181214 10529 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:26.181687 10529 heartbeater.cc:511] Master 127.29.84.190:33257 requested a full tablet report, sending...
I20260806 00:44:26.182621 9849 ts_manager.cc:194] Registered new tserver with Master: 261cbe1dadd04e3e97cba0e6cf49c019 (127.29.84.131:38773)
I20260806 00:44:26.183221 9849 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:33185
I20260806 00:44:26.190274 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab.
Entry for principal oryx/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab.
WARNING: no policy specified for oryx/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab.
Entry for principal oryx/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab.
WARNING: no policy specified for oryx/127.29.84.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab.
Entry for principal oryx/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab.
I20260806 00:44:26.260972 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 9945
I20260806 00:44:26.275705 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 10147
I20260806 00:44:26.288841 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 10343
I20260806 00:44:26.301877 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 9802
I20260806 00:44:26.313982 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:33257
--webserver_interface=127.29.84.190
--webserver_port=41623
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:33257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:26.420554 10544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:26.420845 10544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:26.420919 10544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:26.424901 10544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:26.424983 10544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:26.425022 10544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:26.425076 10544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:26.425112 10544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:26.430181 10544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:33257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:33257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=41623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:26.431161 10544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:26.432126 10544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:26.440335 10553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:26.440335 10555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:26.440366 10552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:26.440547 10544 server_base.cc:1034] running on GCE node
I20260806 00:44:26.441246 10544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:26.442294 10544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:26.443483 10544 hybrid_clock.cc:648] HybridClock initialized: now 1785977066443468 us; error 34 us; skew 500 ppm
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977066, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:26.446178 10544 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:44:26.447000 10544 webserver.cc:460] Webserver started at http://127.29.84.190:41623/ using document root <none> and password file <none>
I20260806 00:44:26.447558 10544 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:26.447672 10544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:26.451494 10544 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:26.453501 10564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.454674 10544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:26.454813 10544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "fb59852e9ab74e4b93373197f9661960"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "1ee450c3d9856f0830912034cdae0c02"
server_key_iv: "c91169fddd13d83c36ee2e237bb8fdc6"
server_key_version: "encryptionkey@0"
I20260806 00:44:26.455403 10544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:26.468601 10544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:26.471764 10544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:26.471956 10544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:26.478837 10544 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:33257
I20260806 00:44:26.478852 10641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:33257 every 8 connection(s)
I20260806 00:44:26.479857 10544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:26.480093 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 10544
I20260806 00:44:26.480615 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:42053
--local_ip_for_outbound_sockets=127.29.84.129
--tserver_master_addrs=127.29.84.190:33257
--webserver_port=46429
--webserver_interface=127.29.84.129
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.84.129
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:44:26.483989 10642 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:44:26.485668 10642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: Bootstrap starting.
I20260806 00:44:26.501876 10642 log.cc:824] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:26.507900 10642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:44:26.508239 10642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: Bootstrap complete.
I20260806 00:44:26.512391 10642 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } }
I20260806 00:44:26.512811 10642 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb59852e9ab74e4b93373197f9661960, State: Initialized, Role: FOLLOWER
I20260806 00:44:26.513319 10642 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } }
I20260806 00:44:26.513458 10642 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:26.513545 10642 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:26.513692 10642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:44:26.515573 10642 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } }
I20260806 00:44:26.515897 10642 leader_election.cc:302] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb59852e9ab74e4b93373197f9661960; no voters:
I20260806 00:44:26.516178 10642 leader_election.cc:288] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:44:26.516307 10647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:44:26.516590 10647 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [term 2 LEADER]: Becoming Leader. State: Replica: fb59852e9ab74e4b93373197f9661960, State: Running, Role: LEADER
I20260806 00:44:26.516953 10647 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } }
I20260806 00:44:26.517347 10642 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:26.517861 10647 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb59852e9ab74e4b93373197f9661960. Latest consensus state: current_term: 2 leader_uuid: "fb59852e9ab74e4b93373197f9661960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } } }
I20260806 00:44:26.517977 10647 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:26.518544 10650 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fb59852e9ab74e4b93373197f9661960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb59852e9ab74e4b93373197f9661960" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33257 } } }
I20260806 00:44:26.518728 10650 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:26.522495 10672 catalog_manager.cc:1259] Loaded cluster ID: 5d0eeb052b884802a510554d6e69c7a0
I20260806 00:44:26.522555 10672 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: loading cluster ID for follower catalog manager: success
I20260806 00:44:26.524886 10672 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: acquiring CA information for follower catalog manager: success
I20260806 00:44:26.526262 10672 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:44:26.526443 10653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:26.527343 10653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:26.527769 10653 catalog_manager.cc:1259] Loaded cluster ID: 5d0eeb052b884802a510554d6e69c7a0
I20260806 00:44:26.527829 10653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:26.528416 10653 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:26.528869 10653 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fb59852e9ab74e4b93373197f9661960: Loaded TSK: 0
I20260806 00:44:26.530015 10653 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:44:26.592095 10645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:26.592305 10645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:26.592382 10645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:26.596282 10645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:26.596329 10645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:26.596426 10645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:26.601395 10645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.129.keytab
--principal=oryx/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:42053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=46429
--tserver_master_addrs=127.29.84.190:33257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:26.602276 10645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:26.603243 10645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:26.611980 10690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:26.611991 10687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:26.612097 10688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:26.612461 10645 server_base.cc:1034] running on GCE node
I20260806 00:44:26.612879 10645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:26.613669 10645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:26.614869 10645 hybrid_clock.cc:648] HybridClock initialized: now 1785977066614833 us; error 47 us; skew 500 ppm
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977066, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:26.618165 10645 init.cc:377] Logged in from keytab as oryx/127.29.84.129@KRBTEST.COM (short username oryx)
I20260806 00:44:26.619232 10645 webserver.cc:460] Webserver started at http://127.29.84.129:46429/ using document root <none> and password file <none>
I20260806 00:44:26.619752 10645 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:26.619859 10645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:26.623617 10645 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:26.625661 10700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.626906 10645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:26.627028 10645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "7eaec7cd60224759a11910bae70793f2"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "2f315969e0dd9b69f8ebb9bceda612e6"
server_key_iv: "e5398efa6842bdce90e015e801b05cb0"
server_key_version: "encryptionkey@0"
I20260806 00:44:26.627580 10645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:26.655339 10645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:26.658697 10645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:26.658892 10645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:26.659546 10645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:26.660353 10645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:26.660425 10645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.660503 10645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:26.660548 10645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.669214 10645 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:42053
I20260806 00:44:26.669257 10862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:42053 every 8 connection(s)
I20260806 00:44:26.670145 10645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:26.677680 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 10645
I20260806 00:44:26.678241 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:34223
--local_ip_for_outbound_sockets=127.29.84.130
--tserver_master_addrs=127.29.84.190:33257
--webserver_port=41369
--webserver_interface=127.29.84.130
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.84.130
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:44:26.682410 10868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:26.671995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:50437 (local address 127.29.84.190:33257)
0806 00:44:26.672419 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:26.672440 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:26.672886 (+ 446us) server_negotiation.cc:407] Connection header received
0806 00:44:26.673853 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:26.673863 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:26.674132 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:26.674391 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:26.675472 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.676386 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.677449 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.677992 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.678176 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.681731 (+ 3555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:50437: BlockingRecv error: recv got EOF from 127.29.84.129:50437 (error 108)
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":135,"threads_started":1}
I20260806 00:44:26.682420 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:26.672063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:50437)
0806 00:44:26.672722 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:26.672773 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:26.673529 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:26.674480 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:26.674487 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:26.674794 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:26.675321 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.675336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.676505 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.676508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.677339 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.677349 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.678154 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.678161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.678363 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.678916 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:26.678942 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:26.681533 (+ 2591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":126,"threads_started":1}
W20260806 00:44:26.682893 10863 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:33257 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.84.190:33257: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:26.688669 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:26.683334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:57463)
0806 00:44:26.683459 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:26.683474 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:26.683550 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:26.683778 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:26.683780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:26.683898 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:26.683988 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.683993 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.685056 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.685058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.685563 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.685570 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.686197 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.686199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.686262 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.686722 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:26.686732 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:26.688583 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 00:44:26.688921 10868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:26.683421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:57463 (local address 127.29.84.190:33257)
0806 00:44:26.683544 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:26.683550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:26.683567 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:44:26.683616 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:26.683621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:26.683673 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:26.683758 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:26.684100 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.684948 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.685681 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.686088 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.686212 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.688775 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:57463: BlockingRecv error: recv got EOF from 127.29.84.129:57463 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:26.694250 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:26.689148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:33443)
0806 00:44:26.689234 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:26.689249 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:26.689324 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:26.689608 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:26.689610 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:26.689724 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:26.689784 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.689788 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.690824 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.690826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.691363 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.691372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.692003 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.692005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.692059 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.692493 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:26.692503 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:26.694174 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 00:44:26.694416 10868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:26.689243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:33443 (local address 127.29.84.190:33257)
0806 00:44:26.689371 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:26.689377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:26.689393 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:44:26.689434 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:26.689438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:26.689487 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:26.689576 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:26.689893 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.690716 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.691478 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.691885 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.692015 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.694318 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:33443: BlockingRecv error: recv got EOF from 127.29.84.129:33443 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 00:44:26.694490 10863 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:44:26.791608 10871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:26.791857 10871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:26.791921 10871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:26.796489 10871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:26.796538 10871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:26.796622 10871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:26.801620 10871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.130.keytab
--principal=oryx/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:34223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=41369
--tserver_master_addrs=127.29.84.190:33257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:26.802524 10871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:26.803504 10871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:26.812590 10880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:26.812574 10879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:26.812567 10882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:26.812841 10871 server_base.cc:1034] running on GCE node
I20260806 00:44:26.813292 10871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:26.813892 10871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:26.815135 10871 hybrid_clock.cc:648] HybridClock initialized: now 1785977066815121 us; error 34 us; skew 500 ppm
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977066, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:26.817909 10871 init.cc:377] Logged in from keytab as oryx/127.29.84.130@KRBTEST.COM (short username oryx)
I20260806 00:44:26.818684 10871 webserver.cc:460] Webserver started at http://127.29.84.130:41369/ using document root <none> and password file <none>
I20260806 00:44:26.819294 10871 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:26.819373 10871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:26.822952 10871 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:44:26.824926 10893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.826020 10871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:26.826134 10871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "cbe89b41da2b42589722abb70313bf61"
format_stamp: "Formatted at 2026-08-06 00:44:25 on dist-test-slave-n326"
server_key: "3359a0efac963d8ddd44043869977b28"
server_key_iv: "3e46386486f3bd6bdd2726230f8b51de"
server_key_version: "encryptionkey@0"
I20260806 00:44:26.826628 10871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:26.852463 10871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:26.855506 10871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:26.855688 10871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:26.856220 10871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:26.856963 10871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:26.857010 10871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.857069 10871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:26.857101 10871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:26.866168 10871 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:34223
I20260806 00:44:26.866209 11060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:34223 every 8 connection(s)
I20260806 00:44:26.867174 10871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:44:26.877123 10868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:26.868863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:35455 (local address 127.29.84.190:33257)
0806 00:44:26.869003 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:26.869007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:26.869972 (+ 965us) server_negotiation.cc:407] Connection header received
0806 00:44:26.870752 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:26.870754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:26.870805 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:26.870872 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:26.871936 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.872434 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.873501 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.873714 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.873791 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.876960 (+ 3169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:35455: BlockingRecv error: recv got EOF from 127.29.84.130:35455 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:44:26.877337 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 10871
I20260806 00:44:26.877655 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:26.869225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:35455)
0806 00:44:26.869826 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:26.869875 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:26.870586 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:26.871024 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:26.871030 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:26.871354 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:26.871802 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.871814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.872543 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.872545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.873393 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.873402 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.873839 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.873842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.873970 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.874454 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:26.874469 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:26.876836 (+ 2367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":109,"threads_started":1}
I20260806 00:44:26.877772 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:38773
--local_ip_for_outbound_sockets=127.29.84.131
--tserver_master_addrs=127.29.84.190:33257
--webserver_port=40969
--webserver_interface=127.29.84.131
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.84.131
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:26.878273 11062 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:33257 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.84.190:33257: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:26.883641 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:26.878694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:52463)
0806 00:44:26.878780 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:26.878795 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:26.878870 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:26.879127 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:26.879129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:26.879284 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:26.879365 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.879370 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.880085 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.880091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.880632 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.880640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.881079 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.881082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.881187 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.881641 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:26.881652 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:26.883512 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 00:44:26.883733 10868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:26.878784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:52463 (local address 127.29.84.190:33257)
0806 00:44:26.878906 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:26.878912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:26.878927 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:44:26.878962 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:26.878965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:26.879007 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:26.879117 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:26.879493 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.879976 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.880750 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.880970 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.881076 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.883620 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:52463: BlockingRecv error: recv got EOF from 127.29.84.130:52463 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:26 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:26.889195 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:26.884087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:55731)
0806 00:44:26.884190 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:26.884203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:26.884273 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:26.884611 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:26.884614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:26.884760 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:26.884821 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.884826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.885523 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.885528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.886032 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:26.886040 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.886473 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.886477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:26.886552 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.887053 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:26.887121 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:26.889056 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:44:26.889381 11062 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:44:26.889292 10868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:26.884343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:55731 (local address 127.29.84.190:33257)
0806 00:44:26.884444 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:26.884447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:26.884456 (+ 9us) server_negotiation.cc:407] Connection header received
0806 00:44:26.884481 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:26.884483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:26.884511 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:26.884597 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:26.884950 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.885419 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.886153 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:26.886356 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:26.886472 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:26.889172 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:55731: BlockingRecv error: recv got EOF from 127.29.84.130:55731 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 00:44:26.986060 11067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:26.986251 11067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:26.986289 11067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:26.990051 11067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:26.990096 11067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:26.990176 11067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:26.995036 11067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/krb5kdc/oryx_127.29.84.131.keytab
--principal=oryx/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:38773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=40969
--tserver_master_addrs=127.29.84.190:33257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:26.995941 11067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:26.996807 11067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:44:27.005632 11067 server_base.cc:1034] running on GCE node
W20260806 00:44:27.005635 11081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:27.005677 11078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:27.005613 11076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:27.006152 11067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:27.006717 11067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:27.007912 11067 hybrid_clock.cc:648] HybridClock initialized: now 1785977067007904 us; error 31 us; skew 500 ppm
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977067, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:27.010658 11067 init.cc:377] Logged in from keytab as oryx/127.29.84.131@KRBTEST.COM (short username oryx)
I20260806 00:44:27.011475 11067 webserver.cc:460] Webserver started at http://127.29.84.131:40969/ using document root <none> and password file <none>
I20260806 00:44:27.011917 11067 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:27.011979 11067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:27.015326 11067 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:27.017269 11088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:27.018515 11067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:27.018635 11067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "261cbe1dadd04e3e97cba0e6cf49c019"
format_stamp: "Formatted at 2026-08-06 00:44:26 on dist-test-slave-n326"
server_key: "43304d7052167ddc3528b220a45a9b1c"
server_key_iv: "ef54b784ff221d2ddde02a242fd93b0a"
server_key_version: "encryptionkey@0"
I20260806 00:44:27.019207 11067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:27.030457 11067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:27.033617 11067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:27.033798 11067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:27.034335 11067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:27.035405 11067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:27.035476 11067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:27.035544 11067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:27.035575 11067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:27.044452 11067 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:38773
I20260806 00:44:27.044515 11253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:38773 every 8 connection(s)
I20260806 00:44:27.045379 11067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:27.054827 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 11067
W20260806 00:44:27.055605 10868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:27.047048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:52595 (local address 127.29.84.190:33257)
0806 00:44:27.047188 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:27.047192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:27.048153 (+ 961us) server_negotiation.cc:407] Connection header received
0806 00:44:27.049017 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:27.049020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:27.049088 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:27.049194 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:27.050265 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.050785 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.051919 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.052150 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.052264 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.055473 (+ 3209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:52595: BlockingRecv error: recv got EOF from 127.29.84.131:52595 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:44:27.056147 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:27.047391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:52595)
0806 00:44:27.048014 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:27.048051 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:27.048806 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:27.049359 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:27.049365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:27.049689 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:27.050119 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.050131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.050924 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.050927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.051755 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.051765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.052301 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.052306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.052466 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.053037 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:27.053052 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:27.055323 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":115,"threads_started":1}
W20260806 00:44:27.056593 11254 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:33257 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.84.190:33257: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:27.062356 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:27.056979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:42511)
0806 00:44:27.057081 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:27.057096 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:27.057167 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:27.057498 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:27.057500 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:27.057660 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:27.057739 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.057744 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.058502 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.058505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.059206 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.059216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.059707 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.059710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.059811 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.060202 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:27.060218 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:27.062227 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:44:27.062471 11261 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:27.057079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:42511 (local address 127.29.84.190:33257)
0806 00:44:27.057319 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:27.057322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:27.057332 (+ 10us) server_negotiation.cc:407] Connection header received
0806 00:44:27.057358 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:27.057360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:27.057390 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:27.057483 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:27.057851 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.058407 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.059318 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.059586 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.059715 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.062337 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:42511: BlockingRecv error: recv got EOF from 127.29.84.131:42511 (error 108)
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":170,"thread_start_us":70,"threads_started":1}
I20260806 00:44:27.065385 10868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:27.055825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.29.84.190:33257)
0806 00:44:27.055940 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:27.055944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:27.056016 (+ 72us) server_negotiation.cc:407] Connection header received
0806 00:44:27.056180 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:27.056183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:27.056227 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:27.056307 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:27.057523 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.058028 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.058941 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.059228 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.059370 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.060992 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:27.061002 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:27.061024 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:27.061060 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:27.063417 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:27.063864 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:27.063870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:27.063875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:27.064007 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:27.064255 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:27.064258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:27.064260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:27.064609 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:27.064751 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:27.065013 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 00:44:27.065247 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":43}
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:27.067945 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:27.062860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:32915)
0806 00:44:27.062957 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:27.062970 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:27.063042 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:27.063292 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:27.063295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:27.063440 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:27.063538 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.063544 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.064357 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.064359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.064970 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.064980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.065635 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.065639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.065694 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.066072 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:27.066082 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:27.067822 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 00:44:27.068122 11254 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:44:27.068238 11261 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:27.062939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:32915 (local address 127.29.84.190:33257)
0806 00:44:27.063098 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:27.063102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:27.063117 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:44:27.063153 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:27.063156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:27.063189 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:27.063278 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:27.063647 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.064262 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.065076 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.065515 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.065610 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.067926 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:32915: BlockingRecv error: recv got EOF from 127.29.84.131:32915 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:27.700359 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:27.695109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:35673)
0806 00:44:27.695408 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:27.695426 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:27.695551 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:27.695784 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:27.695787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:27.695957 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:27.696051 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.696056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.696803 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.696805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.697370 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.697377 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.697813 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.697815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.697875 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.698369 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:27.698380 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:27.700235 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":77,"threads_started":1}
W20260806 00:44:27.700527 11263 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:27.695275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:35673 (local address 127.29.84.190:33257)
0806 00:44:27.695580 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:27.695584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:27.695596 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:44:27.695629 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:27.695631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:27.695670 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:27.695747 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:27.696162 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.696709 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.697483 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.697720 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.697794 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.700368 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:35673: BlockingRecv error: recv got EOF from 127.29.84.129:35673 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":109,"threads_started":1}
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:27 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:27.894883 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:27.889968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:49037)
0806 00:44:27.890243 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:27.890256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:27.890342 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:27.890567 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:27.890569 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:27.890732 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:27.890823 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.890828 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.891558 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.891560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.892103 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:27.892112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.892509 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.892511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:27.892571 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.893031 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:27.893042 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:27.894774 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":100,"threads_started":1}
W20260806 00:44:27.895027 11263 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:27.890064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:49037 (local address 127.29.84.190:33257)
0806 00:44:27.890211 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:27.890214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:27.890305 (+ 91us) server_negotiation.cc:407] Connection header received
0806 00:44:27.890417 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:27.890419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:27.890468 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:27.890530 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:27.890918 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.891467 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.892203 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:27.892407 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:27.892489 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:27.894914 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:49037: BlockingRecv error: recv got EOF from 127.29.84.130:49037 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 00:44:28 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:28 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:28.073860 11266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:28.068654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:39845)
0806 00:44:28.068922 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:28.068936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:28.069019 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:28.069256 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:28.069259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:28.069426 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:28.069518 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:28.069523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.070236 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.070238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:28.070789 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:28.070796 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.071211 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.071214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:28.071275 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:28.071721 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:28.071732 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:28.073751 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":83,"threads_started":1}
W20260806 00:44:28.074015 11263 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:28.068760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:39845 (local address 127.29.84.190:33257)
0806 00:44:28.068914 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:28.068917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:28.068984 (+ 67us) server_negotiation.cc:407] Connection header received
0806 00:44:28.069101 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:28.069103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:28.069148 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:28.069219 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:28.069616 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.070144 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.070890 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.071113 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.071187 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:28.073897 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:39845: BlockingRecv error: recv got EOF from 127.29.84.131:39845 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 00:44:28 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:28 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:28.706750 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:28.701180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:44649)
0806 00:44:28.701504 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:28.701518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:28.701603 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:28.701801 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:28.701803 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:28.701990 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:28.702089 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:28.702094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.702826 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.702828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:28.703411 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:28.703420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.703837 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.703839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:28.703903 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:28.704383 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:28.704395 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:28.706603 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":105,"threads_started":1}
W20260806 00:44:28.706880 11268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:28.701278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:44649 (local address 127.29.84.190:33257)
0806 00:44:28.701554 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:28.701558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:28.701571 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:44:28.701683 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:28.701685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:28.701725 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:28.701790 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:28.702190 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.702730 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.703512 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.703740 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.703817 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:28.706720 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:44649: BlockingRecv error: recv got EOF from 127.29.84.129:44649 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":91,"threads_started":1}
Aug 06 00:44:28 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:28 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:28.901476 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:28.895780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:50181)
0806 00:44:28.896058 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:28.896072 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:28.896167 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:28.896394 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:28.896396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:28.896572 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:28.896670 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:28.896675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.897445 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.897448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:28.898090 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:28.898099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.898514 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.898517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:28.898586 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:28.899101 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:28.899113 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:28.901287 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":93,"threads_started":1}
W20260806 00:44:28.901582 11268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:28.895838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:50181 (local address 127.29.84.190:33257)
0806 00:44:28.896073 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:28.896077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:28.896142 (+ 65us) server_negotiation.cc:407] Connection header received
0806 00:44:28.896244 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:28.896246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:28.896290 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:28.896357 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:28.896765 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.897269 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.898201 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:28.898412 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:28.898485 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:28.901455 (+ 2970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:50181: BlockingRecv error: recv got EOF from 127.29.84.130:50181 (error 108)
Metrics: {"server-negotiator.queue_time_us":152}
Aug 06 00:44:29 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:29 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:29.084774 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:29.074567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:49237)
0806 00:44:29.074850 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:29.074864 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:29.074947 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:29.075200 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:29.075202 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:29.075365 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:29.075460 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:29.075464 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.076148 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.076151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:29.076674 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:29.076681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.077111 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.077113 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:29.077174 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:29.082626 (+ 5452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:29.082645 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:29.084630 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
W20260806 00:44:29.084841 11268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:29.074743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:49237 (local address 127.29.84.190:33257)
0806 00:44:29.074883 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:29.074886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:29.074902 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:44:29.075030 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:29.075032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:29.075094 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:29.075162 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:29.075553 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.076058 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.076780 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.077003 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.077081 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:29.084754 (+ 7673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:49237: BlockingRecv error: recv got EOF from 127.29.84.131:49237 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 00:44:29 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:29 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:29.713063 11274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:29.707525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:47025)
0806 00:44:29.707820 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:29.707833 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:29.707916 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:29.708154 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:29.708156 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:29.708339 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:29.708433 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:29.708437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.709181 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.709183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:29.709840 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:29.709851 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.710434 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.710437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:29.710547 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:29.711170 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:29.711181 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:29.712921 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":122,"threads_started":1}
W20260806 00:44:29.713186 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:29.707710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:47025 (local address 127.29.84.190:33257)
0806 00:44:29.707936 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:29.707940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:29.707951 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:44:29.707993 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:29.707995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:29.708035 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:29.708111 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:29.708533 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.709087 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.709979 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.710312 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.710423 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:29.713062 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:47025: BlockingRecv error: recv got EOF from 127.29.84.129:47025 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":84,"threads_started":1}
Aug 06 00:44:29 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:29 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:29.907588 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:29.902221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:35111)
0806 00:44:29.902490 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:29.902505 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:29.902606 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:29.902833 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:29.902835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:29.903061 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:29.903235 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:29.903243 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.904149 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.904153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:29.904695 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:29.904703 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.905121 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.905124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:29.905183 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:29.905645 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:29.905656 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:29.907479 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":97,"threads_started":1}
W20260806 00:44:29.907775 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:29.902303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:35111 (local address 127.29.84.190:33257)
0806 00:44:29.902435 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:29.902438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:29.902577 (+ 139us) server_negotiation.cc:407] Connection header received
0806 00:44:29.902681 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:29.902683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:29.902727 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:29.902792 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:29.903374 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.904034 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.904798 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:29.905017 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:29.905091 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:29.907625 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:35111: BlockingRecv error: recv got EOF from 127.29.84.130:35111 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 00:44:30 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:30 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:30.091009 11278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:30.085532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:42417)
0806 00:44:30.085822 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:30.085834 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:30.085959 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:30.086251 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:30.086254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:30.086492 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:30.086583 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:30.086589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.087410 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.087413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:30.087952 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:30.087960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.088391 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.088394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:30.088483 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:30.089022 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:30.089034 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:30.090852 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":116,"threads_started":1}
W20260806 00:44:30.091115 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:30.085600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:42417 (local address 127.29.84.190:33257)
0806 00:44:30.085747 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:30.085751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:30.085914 (+ 163us) server_negotiation.cc:407] Connection header received
0806 00:44:30.086078 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:30.086082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:30.086131 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:30.086225 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:30.086703 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.087303 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.088066 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.088284 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.088387 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:30.090977 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:42417: BlockingRecv error: recv got EOF from 127.29.84.131:42417 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 00:44:30 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:30 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:30.719205 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:30.713854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:45433)
0806 00:44:30.714121 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:30.714135 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:30.714221 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:30.714478 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:30.714480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:30.714651 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:30.714745 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:30.714750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.715512 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.715515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:30.716064 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:30.716072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.716485 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.716487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:30.716554 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:30.717099 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:30.717111 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:30.719018 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
W20260806 00:44:30.719329 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:30.714034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:45433 (local address 127.29.84.190:33257)
0806 00:44:30.714288 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:30.714291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:30.714302 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:44:30.714335 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:30.714336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:30.714375 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:30.714448 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:30.714845 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.715422 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.716165 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.716389 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.716462 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:30.719184 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:45433: BlockingRecv error: recv got EOF from 127.29.84.129:45433 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":106,"threads_started":1}
Aug 06 00:44:30 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:30 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:30.913728 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:30.908300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:41685)
0806 00:44:30.908557 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:30.908571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:30.908664 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:30.908890 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:30.908892 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:30.909081 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:30.909185 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:30.909190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.909956 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.909958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:30.910503 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:30.910511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.910928 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.910931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:30.910996 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:30.911543 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:30.911554 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:30.913534 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":89,"threads_started":1}
W20260806 00:44:30.913970 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:30.908362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:41685 (local address 127.29.84.190:33257)
0806 00:44:30.908520 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:30.908525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:30.908665 (+ 140us) server_negotiation.cc:407] Connection header received
0806 00:44:30.908731 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:30.908733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:30.908781 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:30.908850 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:30.909310 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.909857 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.910614 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:30.910830 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:30.910903 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:30.913716 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:41685: BlockingRecv error: recv got EOF from 127.29.84.130:41685 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 00:44:31 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:31 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:31.097424 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:31.091893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:59357)
0806 00:44:31.092174 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:31.092187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:31.092286 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:31.092521 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:31.092523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:31.092710 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:31.092803 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:31.092807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.093489 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.093492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:31.094081 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:31.094089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.094494 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.094497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:31.094559 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:31.095053 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:31.095111 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:31.097313 (+ 2202us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":108,"threads_started":1}
W20260806 00:44:31.097611 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:31.091951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:59357 (local address 127.29.84.190:33257)
0806 00:44:31.092095 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:31.092100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:31.092261 (+ 161us) server_negotiation.cc:407] Connection header received
0806 00:44:31.092365 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:31.092367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:31.092413 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:31.092486 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:31.092895 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.093386 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.094183 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.094400 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.094487 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:31.097447 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:59357: BlockingRecv error: recv got EOF from 127.29.84.131:59357 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 00:44:31 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:31 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:31.725718 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:31.720089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:40405)
0806 00:44:31.720358 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:31.720380 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:31.720476 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:31.720703 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:31.720705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:31.720892 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:31.720987 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:31.720992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.721725 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.721727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:31.722445 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:31.722456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.723113 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.723119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:31.723212 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:31.723741 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:31.723755 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:31.725600 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
W20260806 00:44:31.725894 11285 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:31.720119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:40405 (local address 127.29.84.190:33257)
0806 00:44:31.720361 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:31.720364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:31.720448 (+ 84us) server_negotiation.cc:407] Connection header received
0806 00:44:31.720553 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:31.720555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:31.720604 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:31.720671 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:31.721081 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.721635 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.722583 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.722931 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.723098 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:31.725743 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:40405: BlockingRecv error: recv got EOF from 127.29.84.129:40405 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
Aug 06 00:44:31 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:31 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:31.920639 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:31.914797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:41451)
0806 00:44:31.915126 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:31.915140 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:31.915229 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:31.915447 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:31.915449 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:31.915626 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:31.915724 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:31.915729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.916711 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.916714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:31.917436 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:31.917448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.917939 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.917942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:31.918009 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:31.918584 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:31.918595 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:31.920526 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":108,"threads_started":1}
W20260806 00:44:31.920770 11285 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:31.914921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:41451 (local address 127.29.84.190:33257)
0806 00:44:31.915081 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:31.915084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:31.915205 (+ 121us) server_negotiation.cc:407] Connection header received
0806 00:44:31.915291 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:31.915293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:31.915336 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:31.915401 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:31.915833 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.916563 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.917553 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:31.917828 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:31.917907 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:31.920658 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:41451: BlockingRecv error: recv got EOF from 127.29.84.130:41451 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 00:44:32 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:32 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:32.103699 11287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:32.098202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:47117)
0806 00:44:32.098522 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:32.098537 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:32.098679 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:32.098988 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:32.098991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:32.099279 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:32.099424 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:32.099436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.100180 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.100183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:32.100745 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:32.100753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.101155 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.101157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:32.101214 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:32.101719 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:32.101730 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:32.103580 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":91,"threads_started":1}
W20260806 00:44:32.103863 11285 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:32.098282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:47117 (local address 127.29.84.190:33257)
0806 00:44:32.098424 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:32.098430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:32.098595 (+ 165us) server_negotiation.cc:407] Connection header received
0806 00:44:32.098793 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:32.098797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:32.098862 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:32.098927 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:32.099551 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.100083 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.100849 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.101059 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.101134 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:32.103707 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:47117: BlockingRecv error: recv got EOF from 127.29.84.131:47117 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 00:44:32 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:32 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:32.732151 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:32.726558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:42523)
0806 00:44:32.726863 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:32.726876 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:32.726982 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:32.727242 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:32.727244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:32.727457 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:32.727558 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:32.727564 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.728300 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.728303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:32.728874 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:32.728882 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.729304 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.729306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:32.729426 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:32.729934 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:32.729946 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:32.732039 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":90,"threads_started":1}
W20260806 00:44:32.732323 11289 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:32.726630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:42523 (local address 127.29.84.190:33257)
0806 00:44:32.726872 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:32.726875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:32.726956 (+ 81us) server_negotiation.cc:407] Connection header received
0806 00:44:32.727047 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:32.727049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:32.727107 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:32.727178 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:32.727691 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.728209 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.728972 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.729184 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.729278 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:32.732188 (+ 2910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:42523: BlockingRecv error: recv got EOF from 127.29.84.129:42523 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":60,"threads_started":1}
Aug 06 00:44:32 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:32 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:32.927186 11291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:32.921507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:41357)
0806 00:44:32.921820 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:32.921835 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:32.921979 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:32.922229 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:32.922232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:32.922461 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:32.922603 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:32.922612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.923420 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.923423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:32.924062 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:32.924071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.924487 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.924490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:32.924558 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:32.925101 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:32.925120 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:32.927034 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":114,"threads_started":1}
W20260806 00:44:32.927346 11289 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:32.921581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:41357 (local address 127.29.84.190:33257)
0806 00:44:32.921735 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:32.921738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:32.921908 (+ 170us) server_negotiation.cc:407] Connection header received
0806 00:44:32.922067 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:32.922069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:32.922119 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:32.922185 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:32.922726 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.923271 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.924184 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:32.924390 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:32.924462 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:32.927226 (+ 2764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:41357: BlockingRecv error: recv got EOF from 127.29.84.130:41357 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 00:44:33 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:33 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:33.109447 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:33.104431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:52651)
0806 00:44:33.104695 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:33.104707 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:33.104798 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:33.105026 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:33.105028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:33.105197 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:33.105284 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:33.105289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.106005 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.106007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:33.106527 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:33.106534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.106919 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.106921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:33.106981 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:33.107513 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:33.107532 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:33.109339 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":93,"threads_started":1}
W20260806 00:44:33.109639 11289 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:33.104497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:52651 (local address 127.29.84.190:33257)
0806 00:44:33.104637 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:33.104642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:33.104778 (+ 136us) server_negotiation.cc:407] Connection header received
0806 00:44:33.104879 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:33.104881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:33.104925 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:33.104992 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:33.105403 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.105915 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.106621 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.106826 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.106901 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:33.109509 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:52651: BlockingRecv error: recv got EOF from 127.29.84.131:52651 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 00:44:33 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:33 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:33.738603 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:33.733009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:52047)
0806 00:44:33.733363 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:33.733377 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:33.733473 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:33.733706 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:33.733708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:33.733913 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:33.734028 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:33.734034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.734788 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.734790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:33.735360 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:33.735367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.735757 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.735759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:33.735820 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:33.736392 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:33.736425 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:33.738488 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":88,"threads_started":1}
W20260806 00:44:33.738754 11299 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:33.733114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:52047 (local address 127.29.84.190:33257)
0806 00:44:33.733370 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:33.733373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:33.733447 (+ 74us) server_negotiation.cc:407] Connection header received
0806 00:44:33.733549 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:33.733551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:33.733594 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:33.733668 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:33.734134 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.734693 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.735453 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.735663 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.735737 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:33.738621 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:52047: BlockingRecv error: recv got EOF from 127.29.84.129:52047 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":82,"threads_started":1}
Aug 06 00:44:33 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:33 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:33.933985 11300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:33.928124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:39075)
0806 00:44:33.928440 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:33.928453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:33.928548 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:33.928829 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:33.928831 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:33.929007 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:33.929114 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:33.929119 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.929989 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.929992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:33.930929 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:33.930939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.931468 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.931470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:33.931533 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:33.932060 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:33.932072 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:33.933873 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":112,"threads_started":1}
W20260806 00:44:33.934165 11299 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:33.928219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:39075 (local address 127.29.84.190:33257)
0806 00:44:33.928375 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:33.928403 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:33.928521 (+ 118us) server_negotiation.cc:407] Connection header received
0806 00:44:33.928631 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:33.928634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:33.928686 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:33.928786 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:33.929209 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.929868 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.931036 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:33.931337 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:33.931450 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:33.934011 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:39075: BlockingRecv error: recv got EOF from 127.29.84.130:39075 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 00:44:34 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:34 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:34.119974 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:34.110245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:42771)
0806 00:44:34.110568 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:34.110581 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:34.110687 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:34.111027 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:34.111030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:34.111307 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:34.111415 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:34.111421 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.112292 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.112297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:34.112810 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:34.112817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.113285 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.113289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:34.113373 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:34.117843 (+ 4470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:34.117855 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:34.119805 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":110,"threads_started":1}
W20260806 00:44:34.120039 11299 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:34.110373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:42771 (local address 127.29.84.190:33257)
0806 00:44:34.110509 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:34.110513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:34.110660 (+ 147us) server_negotiation.cc:407] Connection header received
0806 00:44:34.110799 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:34.110804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:34.110862 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:34.110939 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:34.111530 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.112185 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.112924 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.113171 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.113277 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:34.119918 (+ 6641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:42771: BlockingRecv error: recv got EOF from 127.29.84.131:42771 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 00:44:34 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:34 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:34.745376 11302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:34.739568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:54555)
0806 00:44:34.739856 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:34.739875 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:34.739988 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:34.740237 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:34.740239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:34.740425 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:34.740525 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:34.740530 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.741271 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.741275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:34.742009 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:34.742021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.742644 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.742647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:34.742750 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:34.743296 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:34.743308 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:34.745202 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":76,"threads_started":1}
W20260806 00:44:34.745483 11303 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:34.739713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:54555 (local address 127.29.84.190:33257)
0806 00:44:34.739980 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:34.739983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:34.739997 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:44:34.740079 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:34.740081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:34.740124 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:34.740199 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:34.740637 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.741168 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.742142 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.742476 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.742622 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:34.745337 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:54555: BlockingRecv error: recv got EOF from 127.29.84.129:54555 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
Aug 06 00:44:34 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:34 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:34.940104 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:34.934855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:43621)
0806 00:44:34.935135 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:34.935148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:34.935253 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:34.935497 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:34.935499 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:34.935676 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:34.935776 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:34.935781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.936629 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.936632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:34.937177 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:34.937184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.937576 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.937578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:34.937637 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:34.938104 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:34.938115 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:34.939989 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":90,"threads_started":1}
W20260806 00:44:34.940275 11303 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:34.934908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:43621 (local address 127.29.84.190:33257)
0806 00:44:34.935045 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:34.935048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:34.935223 (+ 175us) server_negotiation.cc:407] Connection header received
0806 00:44:34.935326 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:34.935328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:34.935372 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:34.935445 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:34.935873 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.936536 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.937273 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:34.937484 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:34.937556 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:34.940121 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:43621: BlockingRecv error: recv got EOF from 127.29.84.130:43621 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 00:44:35 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:35 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:35.125926 11306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:35.120734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:53501)
0806 00:44:35.120984 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:35.120997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:35.121087 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:35.121306 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:35.121308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:35.121523 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:35.121618 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:35.121623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.122327 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.122329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:35.122886 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:35.122894 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.123393 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.123396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:35.123462 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:35.123975 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:35.123988 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:35.125815 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":89,"threads_started":1}
W20260806 00:44:35.126088 11303 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:35.120806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:53501 (local address 127.29.84.190:33257)
0806 00:44:35.120933 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:35.120936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:35.121060 (+ 124us) server_negotiation.cc:407] Connection header received
0806 00:44:35.121162 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:35.121164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:35.121209 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:35.121274 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:35.121721 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.122233 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.123011 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.123304 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.123384 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:35.125944 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:53501: BlockingRecv error: recv got EOF from 127.29.84.131:53501 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 00:44:35 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:35 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:35.751642 11307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:35.746241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:40209)
0806 00:44:35.746543 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:35.746556 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:35.746654 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:35.746880 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:35.746882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:35.747072 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:35.747237 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:35.747244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.747952 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.747954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:35.748508 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:35.748514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.748903 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.748906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:35.748966 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:35.749537 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:35.749550 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:35.751480 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":88,"threads_started":1}
W20260806 00:44:35.751787 11308 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:35.746364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:40209 (local address 127.29.84.190:33257)
0806 00:44:35.746583 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:35.746587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:35.746610 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:44:35.746743 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:35.746745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:35.746796 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:35.746870 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:35.747343 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.747861 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.748601 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.748810 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.748882 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:35.751612 (+ 2730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:40209: BlockingRecv error: recv got EOF from 127.29.84.129:40209 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":58,"threads_started":1}
Aug 06 00:44:35 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:35 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:35.946542 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:35.940851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:39175)
0806 00:44:35.941147 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:35.941169 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:35.941308 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:35.941603 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:35.941606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:35.941812 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:35.941912 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:35.941918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.942697 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.942701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:35.943320 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:35.943330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.943787 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.943790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:35.943881 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:35.944459 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:35.944471 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:35.946354 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":99,"threads_started":1}
W20260806 00:44:35.946569 11308 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:35.940926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:39175 (local address 127.29.84.190:33257)
0806 00:44:35.941114 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:35.941119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:35.941248 (+ 129us) server_negotiation.cc:407] Connection header received
0806 00:44:35.941422 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:35.941426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:35.941481 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:35.941568 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:35.942033 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.942572 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.943439 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:35.943667 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:35.943779 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:35.946470 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:39175: BlockingRecv error: recv got EOF from 127.29.84.130:39175 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 00:44:36 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:36 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:36.132541 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:36.126712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:33477)
0806 00:44:36.127010 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:36.127024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:36.127153 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:36.127368 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:36.127370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:36.127553 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:36.127650 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:36.127656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.128418 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.128421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:36.128979 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:36.128986 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.129577 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.129586 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:36.129674 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:36.130265 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:36.130280 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:36.132409 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":98,"threads_started":1}
W20260806 00:44:36.132656 11308 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:36.126779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:33477 (local address 127.29.84.190:33257)
0806 00:44:36.126922 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:36.126927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:36.127110 (+ 183us) server_negotiation.cc:407] Connection header received
0806 00:44:36.127219 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:36.127221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:36.127266 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:36.127330 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:36.127746 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.128274 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.129103 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.129430 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.129555 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:36.132532 (+ 2977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:33477: BlockingRecv error: recv got EOF from 127.29.84.131:33477 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 00:44:36 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:36 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:36.758023 11312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:36.752466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:32913)
0806 00:44:36.752741 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:36.752755 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:36.752884 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:36.753186 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:36.753189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:36.753396 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:36.753556 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:36.753565 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.754556 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.754559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:36.755190 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:36.755199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.755616 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.755618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:36.755682 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:36.756148 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:36.756160 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:36.757908 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":92,"threads_started":1}
W20260806 00:44:36.758191 11313 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:36.752500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:32913 (local address 127.29.84.190:33257)
0806 00:44:36.752753 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:36.752756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:36.752824 (+ 68us) server_negotiation.cc:407] Connection header received
0806 00:44:36.752985 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:36.752988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:36.753032 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:36.753133 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:36.753685 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.754450 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.755286 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.755519 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.755593 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:36.758055 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:32913: BlockingRecv error: recv got EOF from 127.29.84.129:32913 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
Aug 06 00:44:36 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:36 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:36.953169 11315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:36.947471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:54121)
0806 00:44:36.947810 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:36.947828 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:36.947964 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:36.948250 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:36.948253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:36.948480 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:36.948574 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:36.948579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.949354 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.949358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:36.949936 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:36.949944 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.950420 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.950424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:36.950511 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:36.951030 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:36.951041 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:36.952995 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":120,"threads_started":1}
W20260806 00:44:36.953246 11313 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:36.947665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:54121 (local address 127.29.84.190:33257)
0806 00:44:36.947811 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:36.947818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:36.947914 (+ 96us) server_negotiation.cc:407] Connection header received
0806 00:44:36.948068 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:36.948072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:36.948136 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:36.948222 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:36.948707 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.949245 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.950063 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:36.950296 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:36.950415 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:36.953136 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:54121: BlockingRecv error: recv got EOF from 127.29.84.130:54121 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 00:44:37 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:37 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:37.138489 11316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:37.133286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:33217)
0806 00:44:37.133593 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:37.133607 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:37.133703 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:37.133942 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:37.133944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:37.134129 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:37.134224 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:37.134229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.134908 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.134910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:37.135482 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:37.135490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.135892 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.135895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:37.135953 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:37.136443 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:37.136455 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:37.138345 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":107,"threads_started":1}
W20260806 00:44:37.138670 11313 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:37.133439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:33217 (local address 127.29.84.190:33257)
0806 00:44:37.133599 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:37.133602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:37.133677 (+ 75us) server_negotiation.cc:407] Connection header received
0806 00:44:37.133781 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:37.133783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:37.133828 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:37.133910 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:37.134317 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.134824 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.135582 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.135792 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.135874 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:37.138526 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:33217: BlockingRecv error: recv got EOF from 127.29.84.131:33217 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 00:44:37 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:37 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:37.764415 11318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:37.758863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:45605)
0806 00:44:37.759213 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:37.759226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:37.759319 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:37.759609 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:37.759612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:37.759824 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:37.759935 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:37.759942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.760676 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.760678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:37.761255 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:37.761263 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.761700 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.761702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:37.761775 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:37.762341 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:37.762362 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:37.764299 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":107,"threads_started":1}
W20260806 00:44:37.764606 11321 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:37.758931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:45605 (local address 127.29.84.190:33257)
0806 00:44:37.759209 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:37.759212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:37.759271 (+ 59us) server_negotiation.cc:407] Connection header received
0806 00:44:37.759410 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:37.759413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:37.759465 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:37.759555 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:37.760041 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.760576 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.761352 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.761602 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.761677 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:37.764431 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:45605: BlockingRecv error: recv got EOF from 127.29.84.129:45605 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":67,"threads_started":1}
Aug 06 00:44:37 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:37 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:37.959666 11325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:37.954123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:58609)
0806 00:44:37.954422 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:37.954438 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:37.954540 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:37.954774 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:37.954776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:37.954983 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:37.955160 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:37.955169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.955923 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.955926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:37.956520 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:37.956528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.956927 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.956929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:37.956996 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:37.957487 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:37.957500 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:37.959497 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":109,"threads_started":1}
W20260806 00:44:37.959810 11321 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:37.954205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:58609 (local address 127.29.84.190:33257)
0806 00:44:37.954344 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:37.954348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:37.954491 (+ 143us) server_negotiation.cc:407] Connection header received
0806 00:44:37.954624 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:37.954627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:37.954667 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:37.954725 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:37.955266 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.955821 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.956620 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:37.956821 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:37.956892 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:37.959659 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:58609: BlockingRecv error: recv got EOF from 127.29.84.130:58609 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 00:44:38 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:38 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:38.144966 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:38.139422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:39599)
0806 00:44:38.139708 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:38.139722 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:38.139822 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:38.140056 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:38.140058 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:38.140250 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:38.140345 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:38.140350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.141055 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.141057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:38.141639 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:38.141648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.142184 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.142188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:38.142289 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:38.142940 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:38.142959 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:38.144841 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":96,"threads_started":1}
W20260806 00:44:38.145109 11321 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:38.139476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:39599 (local address 127.29.84.190:33257)
0806 00:44:38.139622 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:38.139627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:38.139797 (+ 170us) server_negotiation.cc:407] Connection header received
0806 00:44:38.139902 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:38.139904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:38.139952 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:38.140018 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:38.140442 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.140953 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.141796 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.142050 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.142171 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:38.144976 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:39599: BlockingRecv error: recv got EOF from 127.29.84.131:39599 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 00:44:38 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:38 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:38.770691 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:38.765139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:33931)
0806 00:44:38.765406 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:38.765418 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:38.765516 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:38.765740 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:38.765742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:38.765917 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:38.766014 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:38.766019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.766739 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.766742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:38.767585 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:38.767598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.768157 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.768160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:38.768222 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:38.768756 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:38.768768 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:38.770573 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":82,"threads_started":1}
W20260806 00:44:38.770845 11328 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:38.765209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:33931 (local address 127.29.84.190:33257)
0806 00:44:38.765432 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:38.765436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:38.765482 (+ 46us) server_negotiation.cc:407] Connection header received
0806 00:44:38.765588 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:38.765590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:38.765633 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:38.765710 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:38.766106 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.766619 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.767715 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.768055 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.768135 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:38.770712 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:33931: BlockingRecv error: recv got EOF from 127.29.84.129:33931 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
Aug 06 00:44:38 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:38 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:38.965693 11329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:38.960501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:52953)
0806 00:44:38.960758 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:38.960772 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:38.960866 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:38.961094 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:38.961097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:38.961274 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:38.961384 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:38.961389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.962102 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.962105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:38.962709 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:38.962717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.963154 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.963157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:38.963221 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:38.963717 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:38.963730 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:38.965548 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
W20260806 00:44:38.965831 11328 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:38.960575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:52953 (local address 127.29.84.190:33257)
0806 00:44:38.960715 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:38.960718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:38.960834 (+ 116us) server_negotiation.cc:407] Connection header received
0806 00:44:38.960932 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:38.960934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:38.960990 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:38.961052 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:38.961514 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.962014 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.962805 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:38.963011 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:38.963117 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:38.965709 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:52953: BlockingRecv error: recv got EOF from 127.29.84.130:52953 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 00:44:39 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:39 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:39.155938 11331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:39.145911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:57299)
0806 00:44:39.146194 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:39.146207 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:39.146305 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:39.146557 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:39.146559 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:39.146743 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:39.146841 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:39.146847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.147607 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.147610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:39.148166 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:39.148173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.148556 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.148559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:39.148620 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:39.153944 (+ 5324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:39.153958 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:39.155796 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
W20260806 00:44:39.156067 11328 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:39.146001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:57299 (local address 127.29.84.190:33257)
0806 00:44:39.146152 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:39.146155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:39.146279 (+ 124us) server_negotiation.cc:407] Connection header received
0806 00:44:39.146379 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:39.146381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:39.146426 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:39.146494 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:39.146947 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.147518 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.148260 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.148466 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.148539 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:39.155942 (+ 7403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:57299: BlockingRecv error: recv got EOF from 127.29.84.131:57299 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 00:44:39 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:39 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:39.777011 11334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:39.771491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:33079)
0806 00:44:39.771779 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:39.771792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:39.771887 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:39.772113 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:39.772115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:39.772288 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:39.772383 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:39.772388 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.773127 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.773129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:39.773791 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:39.773801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.774408 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.774412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:39.774499 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:39.775085 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:39.775098 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:39.776889 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":90,"threads_started":1}
W20260806 00:44:39.777135 11335 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:39.771549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:33079 (local address 127.29.84.190:33257)
0806 00:44:39.771806 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:39.771810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:39.771840 (+ 30us) server_negotiation.cc:407] Connection header received
0806 00:44:39.771962 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:39.771964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:39.772007 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:39.772081 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:39.772487 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.773038 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.773908 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.774210 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.774412 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:39.777013 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:33079: BlockingRecv error: recv got EOF from 127.29.84.129:33079 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":66,"threads_started":1}
Aug 06 00:44:39 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:39 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:39.972121 11337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:39.966533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:34007)
0806 00:44:39.966805 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:39.966820 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:39.966939 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:39.967238 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:39.967241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:39.967472 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:39.967571 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:39.967577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.968347 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.968351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:39.968918 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:39.968926 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.969368 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.969371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:39.969474 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:39.970077 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:39.970089 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:39.971940 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":98,"threads_started":1}
W20260806 00:44:39.972158 11335 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:39.966637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:34007 (local address 127.29.84.190:33257)
0806 00:44:39.966780 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:39.966785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:39.966902 (+ 117us) server_negotiation.cc:407] Connection header received
0806 00:44:39.967058 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:39.967077 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:39.967130 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:39.967209 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:39.967694 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.968238 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.969039 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:39.969262 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:39.969356 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:39.972058 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:34007: BlockingRecv error: recv got EOF from 127.29.84.130:34007 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 00:44:40 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:40 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:40.162611 11338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:40.156903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:51103)
0806 00:44:40.157192 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:40.157207 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:40.157335 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:40.157623 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:40.157626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:40.157836 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:40.157939 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:40.157945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.158839 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.158844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:40.159534 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:40.159544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.160007 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.160011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:40.160099 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:40.160561 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:40.160573 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:40.162429 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
W20260806 00:44:40.162657 11335 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:40.156946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:51103 (local address 127.29.84.190:33257)
0806 00:44:40.157132 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:40.157138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:40.157284 (+ 146us) server_negotiation.cc:407] Connection header received
0806 00:44:40.157467 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:40.157469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:40.157517 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:40.157599 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:40.158076 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.158726 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.159652 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.159894 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.160002 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:40.162550 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:51103: BlockingRecv error: recv got EOF from 127.29.84.131:51103 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 00:44:40 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:40 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:40.783728 11339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:40.778129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:58367)
0806 00:44:40.778417 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:40.778431 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:40.778522 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:40.778756 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:40.778758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:40.778935 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:40.779036 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:40.779041 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.779867 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.779871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:40.780468 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:40.780477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.780945 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.780949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:40.781045 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:40.781660 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:40.781673 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:40.783535 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":100,"threads_started":1}
W20260806 00:44:40.783795 11340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:40.778201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:58367 (local address 127.29.84.190:33257)
0806 00:44:40.778496 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:40.778499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:40.778513 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:44:40.778605 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:40.778607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:40.778650 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:40.778721 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:40.779205 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.779755 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.780596 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.780827 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.780930 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:40.783664 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:58367: BlockingRecv error: recv got EOF from 127.29.84.129:58367 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":84,"threads_started":1}
Aug 06 00:44:40 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:40 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:40.978170 11343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:40.972921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:49953)
0806 00:44:40.973241 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:40.973257 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:40.973374 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:40.973649 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:40.973651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:40.973847 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:40.973966 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:40.973971 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.974653 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.974655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:40.975224 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:40.975232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.975651 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.975654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:40.975717 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:40.976276 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:40.976288 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:40.978045 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":138,"threads_started":1}
W20260806 00:44:40.978294 11340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:40.972960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:49953 (local address 127.29.84.190:33257)
0806 00:44:40.973131 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:40.973137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:40.973316 (+ 179us) server_negotiation.cc:407] Connection header received
0806 00:44:40.973483 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:40.973489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:40.973560 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:40.973626 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:40.974060 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.974568 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.975320 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:40.975558 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:40.975633 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:40.978181 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:49953: BlockingRecv error: recv got EOF from 127.29.84.130:49953 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 00:44:41 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:41 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.131@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:41.168942 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:41.163474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.131:33037)
0806 00:44:41.163774 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:41.163787 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:41.163886 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:41.164108 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:41.164110 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:41.164291 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:41.164384 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:41.164388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.165073 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.165075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:41.165754 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:41.165764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.166222 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.166226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:41.166304 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:41.166816 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:41.166827 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:41.168763 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":125,"threads_started":1}
W20260806 00:44:41.168977 11340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:41.163522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:33037 (local address 127.29.84.190:33257)
0806 00:44:41.163669 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:41.163674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:41.163857 (+ 183us) server_negotiation.cc:407] Connection header received
0806 00:44:41.163950 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:41.163952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:41.164001 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:41.164067 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:41.164485 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.164987 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.165874 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.166125 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.166202 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:41.168873 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.131:33037: BlockingRecv error: recv got EOF from 127.29.84.131:33037 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 00:44:41 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:41 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.129@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:41.790052 11345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:41.784753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.129:48585)
0806 00:44:41.785048 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:41.785061 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:41.785146 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:41.785389 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:41.785391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:41.785579 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:41.785686 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:41.785692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.786366 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.786369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:41.786971 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:41.786978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.787422 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.787424 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:41.787504 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:41.788004 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:41.788015 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:41.789889 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
W20260806 00:44:41.790310 11347 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:41.784883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:48585 (local address 127.29.84.190:33257)
0806 00:44:41.785110 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:41.785113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:41.785126 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:44:41.785233 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:41.785235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:41.785278 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:41.785352 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:41.785778 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.786281 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.787105 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.787331 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.787406 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:41.790058 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.129:48585: BlockingRecv error: recv got EOF from 127.29.84.129:48585 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":61,"threads_started":1}
Aug 06 00:44:41 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:44:41 dist-test-slave-n326 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.84.130@KRBTEST.COM for oryx/127.29.84.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:44:41.984805 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:41.979177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33257 (local address 127.29.84.130:56807)
0806 00:44:41.979465 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:41.979478 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:41.979596 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:41.979859 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:41.979862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:41.980102 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:41.980202 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:41.980208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.980969 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.980974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:41.981720 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:41.981731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.982169 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.982172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:41.982261 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:41.982835 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:41.982855 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:41.984629 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.84.190:33257: Server oryx/127.29.84.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":95,"threads_started":1}
W20260806 00:44:41.984860 11347 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:41.979231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:56807 (local address 127.29.84.190:33257)
0806 00:44:41.979406 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:41.979411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:41.979550 (+ 139us) server_negotiation.cc:407] Connection header received
0806 00:44:41.979696 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:41.979700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:41.979749 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:41.979832 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:41.980323 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.980859 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.981831 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:41.982050 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:41.982155 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:41.984751 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.84.130:56807: BlockingRecv error: recv got EOF from 127.29.84.130:56807 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 00:44:42.060325 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 10645
I20260806 00:44:42.072398 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 10871
I20260806 00:44:42.082118 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 11067
I20260806 00:44:42.093772 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 10544
2026-08-06T00:44:42Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18117 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T00:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:42Z Disabled control of system clock
I20260806 00:44:42.127899 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:42553
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:42423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 00:44:42.236141 11352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:42.236347 11352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:42.236378 11352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:42.240061 11352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:42.240125 11352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:42.240141 11352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:42.240159 11352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:42.240173 11352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:42.244702 11352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:42423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:42553
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:42553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:42.245513 11352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:42.246672 11352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:42.254547 11361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:42.254554 11359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:42.254609 11365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:42.254746 11352 server_base.cc:1034] running on GCE node
I20260806 00:44:42.255448 11352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:42.256410 11352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:42.257593 11352 hybrid_clock.cc:648] HybridClock initialized: now 1785977082257573 us; error 41 us; skew 500 ppm
I20260806 00:44:42.259282 11352 webserver.cc:460] Webserver started at http://127.29.84.190:42541/ using document root <none> and password file <none>
I20260806 00:44:42.259821 11352 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:42.259910 11352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:42.260128 11352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:42.262008 11352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "be5998fc46814ab9a5938b47157de18a"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "dbfb5114edf65805a9679d8be10e51cd"
server_key_iv: "77cff061c0077ec6887fc8ad2912b096"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.262596 11352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "be5998fc46814ab9a5938b47157de18a"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "dbfb5114edf65805a9679d8be10e51cd"
server_key_iv: "77cff061c0077ec6887fc8ad2912b096"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.265990 11352 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:44:42.268209 11374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.269306 11352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:42.269455 11352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "be5998fc46814ab9a5938b47157de18a"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "dbfb5114edf65805a9679d8be10e51cd"
server_key_iv: "77cff061c0077ec6887fc8ad2912b096"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.269569 11352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:42.284730 11352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:42.285279 11352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:42.285466 11352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:42.291914 11352 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:42553
I20260806 00:44:42.291946 11448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:42553 every 8 connection(s)
I20260806 00:44:42.292785 11352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:42.295701 11449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:42.300562 11449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a: Bootstrap starting.
I20260806 00:44:42.302745 11449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:42.302876 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 11352
I20260806 00:44:42.303010 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:42.303304 30034 external_mini_cluster.cc:1428] Setting key f1d17b3ec7dc722f834db7a1cb247be7
I20260806 00:44:42.303622 11449 log.cc:824] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:42.305685 11449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a: No bootstrap required, opened a new log
I20260806 00:44:42.309715 11449 raft_consensus.cc:348] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5998fc46814ab9a5938b47157de18a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42553 } }
I20260806 00:44:42.309950 11449 raft_consensus.cc:374] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:42.310012 11449 raft_consensus.cc:729] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be5998fc46814ab9a5938b47157de18a, State: Initialized, Role: FOLLOWER
I20260806 00:44:42.310461 11449 consensus_queue.cc:260] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5998fc46814ab9a5938b47157de18a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42553 } }
I20260806 00:44:42.310588 11449 raft_consensus.cc:388] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:42.310675 11449 raft_consensus.cc:482] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:42.310774 11449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:42.311722 11449 raft_consensus.cc:504] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5998fc46814ab9a5938b47157de18a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42553 } }
I20260806 00:44:42.312018 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:42.304370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57652 (local address 127.29.84.190:42553)
0806 00:44:42.304966 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:42.304987 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:42.305017 (+ 30us) server_negotiation.cc:407] Connection header received
0806 00:44:42.305607 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:42.305623 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:42.305977 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:42.306230 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:42.306536 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.307557 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.308815 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.309246 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.309357 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:42.310209 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:42.310214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:42.310225 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:42.310610 (+ 385us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:42.310695 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:42.310722 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:42.310815 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:42.310979 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 00:44:42.311274 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":155,"threads_started":1}
I20260806 00:44:42.312039 11449 leader_election.cc:302] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be5998fc46814ab9a5938b47157de18a; no voters:
I20260806 00:44:42.312371 11449 leader_election.cc:288] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:42.312486 11457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:42.312704 11457 raft_consensus.cc:686] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [term 1 LEADER]: Becoming Leader. State: Replica: be5998fc46814ab9a5938b47157de18a, State: Running, Role: LEADER
I20260806 00:44:42.313040 11457 consensus_queue.cc:237] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5998fc46814ab9a5938b47157de18a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42553 } }
I20260806 00:44:42.313503 11449 sys_catalog.cc:564] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:42.314098 11457 sys_catalog.cc:455] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [sys.catalog]: SysCatalogTable state changed. Reason: New leader be5998fc46814ab9a5938b47157de18a. Latest consensus state: current_term: 1 leader_uuid: "be5998fc46814ab9a5938b47157de18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5998fc46814ab9a5938b47157de18a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42553 } } }
I20260806 00:44:42.314319 11457 sys_catalog.cc:458] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:42.314457 11459 sys_catalog.cc:455] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be5998fc46814ab9a5938b47157de18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5998fc46814ab9a5938b47157de18a" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42553 } } }
I20260806 00:44:42.314565 11459 sys_catalog.cc:458] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:42.314779 11468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:42.317790 11468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:42.323006 11468 catalog_manager.cc:1347] Generated new cluster ID: 0af3506d305041078147842b089ecf71
I20260806 00:44:42.323100 11468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:42.340123 11468 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:42.341180 11468 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:42.348112 11468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be5998fc46814ab9a5938b47157de18a: Generated new TSK 0
I20260806 00:44:42.348790 11468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:44:42.353374 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42553
--builtin_ntp_servers=127.29.84.148:42423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 00:44:42.460659 11484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:42.460898 11484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:42.460958 11484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:42.464901 11484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:42.464969 11484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:42.465041 11484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:42.469587 11484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:42423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:42.470489 11484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:42.471796 11484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:42.481196 11496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:42.481376 11484 server_base.cc:1034] running on GCE node
W20260806 00:44:42.481233 11493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:42.481199 11494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:42.481851 11484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:42.482439 11484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:42.483624 11484 hybrid_clock.cc:648] HybridClock initialized: now 1785977082483605 us; error 39 us; skew 500 ppm
I20260806 00:44:42.485388 11484 webserver.cc:460] Webserver started at http://127.29.84.129:35705/ using document root <none> and password file <none>
I20260806 00:44:42.485931 11484 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:42.486027 11484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:42.486244 11484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:42.488173 11484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "3c5672a15975485cbdbdb25d43d7d5c3"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "868308ebfabf1bf356707ab0d21fcb31"
server_key_iv: "ae45260d83819c1681f121ad9b4d2953"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.488722 11484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "3c5672a15975485cbdbdb25d43d7d5c3"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "868308ebfabf1bf356707ab0d21fcb31"
server_key_iv: "ae45260d83819c1681f121ad9b4d2953"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.492223 11484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:42.494481 11504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.495584 11484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:42.495718 11484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "3c5672a15975485cbdbdb25d43d7d5c3"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "868308ebfabf1bf356707ab0d21fcb31"
server_key_iv: "ae45260d83819c1681f121ad9b4d2953"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.495826 11484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:42.514557 11484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:42.515172 11484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:42.515349 11484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:42.515959 11484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:42.516824 11484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:42.516894 11484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.516958 11484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:42.516988 11484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.525493 11484 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:43869
I20260806 00:44:42.525556 11669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:43869 every 8 connection(s)
I20260806 00:44:42.526324 11484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:42.529825 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 11484
I20260806 00:44:42.529943 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:42.530220 30034 external_mini_cluster.cc:1428] Setting key aca922c1d09531d97c5a509af835e11b
I20260806 00:44:42.532083 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42553
--builtin_ntp_servers=127.29.84.148:42423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 00:44:42.535228 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:42.528017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:34315 (local address 127.29.84.190:42553)
0806 00:44:42.528139 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:42.528143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:42.529060 (+ 917us) server_negotiation.cc:407] Connection header received
0806 00:44:42.530574 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:42.530578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:42.530640 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:42.530742 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:42.531618 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.532155 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.533385 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.533607 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.533711 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:42.534706 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:42.534710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:42.534712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:42.534730 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:42.534749 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:42.534756 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:42.534846 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:42.535006 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 00:44:42.535091 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:44:42.536016 11677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:42.528316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:42553 (local address 127.29.84.129:34315)
0806 00:44:42.528917 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:42.528953 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:42.530381 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:42.530908 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:42.530914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:42.530996 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:42.531483 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:42.531498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.532361 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.532364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:42.533246 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:42.533256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.533757 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.533761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:42.533911 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:42.534473 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:42.534475 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:42.534512 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:42.534567 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:42.534861 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:42.534875 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:42.534881 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:44:42.535000 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 00:44:42.535290 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":112,"threads_started":1}
I20260806 00:44:42.537299 11672 heartbeater.cc:347] Connected to a master server at 127.29.84.190:42553
I20260806 00:44:42.537521 11672 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:42.537968 11672 heartbeater.cc:511] Master 127.29.84.190:42553 requested a full tablet report, sending...
I20260806 00:44:42.539608 11398 ts_manager.cc:194] Registered new tserver with Master: 3c5672a15975485cbdbdb25d43d7d5c3 (127.29.84.129:43869)
I20260806 00:44:42.541316 11398 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.129:34315
W20260806 00:44:42.642203 11678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:42.642423 11678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:42.642458 11678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:42.646313 11678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:42.646354 11678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:42.646404 11678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:42.650877 11678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:42423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:42.651803 11678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:42.652952 11678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:42.662171 11687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:42.662176 11690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:42.662240 11688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:42.662609 11678 server_base.cc:1034] running on GCE node
I20260806 00:44:42.662976 11678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:42.663573 11678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:42.664736 11678 hybrid_clock.cc:648] HybridClock initialized: now 1785977082664720 us; error 25 us; skew 500 ppm
I20260806 00:44:42.666380 11678 webserver.cc:460] Webserver started at http://127.29.84.130:33665/ using document root <none> and password file <none>
I20260806 00:44:42.666906 11678 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:42.666966 11678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:42.667235 11678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:42.669113 11678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "81f8471d71f247f79fea2d3684c52f1a"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "9a720c8a082de23a31403ae693812b1e"
server_key_iv: "73b5c6fc8a6bed0ad8917a7532a22154"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.669621 11678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "81f8471d71f247f79fea2d3684c52f1a"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "9a720c8a082de23a31403ae693812b1e"
server_key_iv: "73b5c6fc8a6bed0ad8917a7532a22154"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.673173 11678 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:44:42.675482 11698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.676601 11678 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:42.676715 11678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "81f8471d71f247f79fea2d3684c52f1a"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "9a720c8a082de23a31403ae693812b1e"
server_key_iv: "73b5c6fc8a6bed0ad8917a7532a22154"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.676811 11678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:42.693866 11678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:42.694463 11678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:42.694631 11678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:42.695195 11678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:42.696017 11678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:42.696063 11678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.696123 11678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:42.696159 11678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.704742 11678 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:33659
I20260806 00:44:42.704789 11876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:33659 every 8 connection(s)
I20260806 00:44:42.705516 11678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:42.708731 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 11678
I20260806 00:44:42.708825 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:42.709102 30034 external_mini_cluster.cc:1428] Setting key b05826a02207c8101b6a10ccb9ab0134
I20260806 00:44:42.711220 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42553
--builtin_ntp_servers=127.29.84.148:42423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 00:44:42.715577 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:42.707040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:43203 (local address 127.29.84.190:42553)
0806 00:44:42.707193 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:42.707197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:42.708088 (+ 891us) server_negotiation.cc:407] Connection header received
0806 00:44:42.709506 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:42.709509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:42.709569 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:42.709695 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:42.710744 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.711594 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.713335 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.713743 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.713867 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:42.714933 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:42.714940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:42.714945 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:42.714960 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:42.714986 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:42.714999 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:42.715089 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:42.715267 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 00:44:42.715337 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:44:42.716127 11881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:42.707401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:42553 (local address 127.29.84.130:43203)
0806 00:44:42.707977 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:42.708012 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:42.709311 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:42.709884 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:42.709904 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:42.710000 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:42.710598 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:42.710623 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.711848 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.711854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:42.713218 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:42.713229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.713910 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.713913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:42.714042 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:42.714685 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:42.714687 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:42.714722 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:42.714780 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:42.715106 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:42.715115 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:42.715125 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 00:44:42.715223 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 00:44:42.715482 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":107,"threads_started":1}
I20260806 00:44:42.717175 11878 heartbeater.cc:347] Connected to a master server at 127.29.84.190:42553
I20260806 00:44:42.717418 11878 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:42.718034 11878 heartbeater.cc:511] Master 127.29.84.190:42553 requested a full tablet report, sending...
I20260806 00:44:42.719146 11398 ts_manager.cc:194] Registered new tserver with Master: 81f8471d71f247f79fea2d3684c52f1a (127.29.84.130:33659)
I20260806 00:44:42.719682 11398 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.130:43203
W20260806 00:44:42.821496 11882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:42.821704 11882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:42.821766 11882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:42.825649 11882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:42.825693 11882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:42.825791 11882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:42.830181 11882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:42423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:42.831051 11882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:42.832376 11882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:42.841377 11891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:42.841508 11882 server_base.cc:1034] running on GCE node
W20260806 00:44:42.841360 11893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:42.841361 11890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:42.842042 11882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:42.842608 11882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:42.843760 11882 hybrid_clock.cc:648] HybridClock initialized: now 1785977082843747 us; error 25 us; skew 500 ppm
I20260806 00:44:42.845356 11882 webserver.cc:460] Webserver started at http://127.29.84.131:33915/ using document root <none> and password file <none>
I20260806 00:44:42.845853 11882 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:42.845921 11882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:42.846084 11882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:42.847976 11882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "e6d7dffcff3140fa8f810f23dba66cc3"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "91260bb2d098229f53c40244d55a20b9"
server_key_iv: "165a3f2f336a2854cbeef11c8f37fc5f"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.848526 11882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "e6d7dffcff3140fa8f810f23dba66cc3"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "91260bb2d098229f53c40244d55a20b9"
server_key_iv: "165a3f2f336a2854cbeef11c8f37fc5f"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.852053 11882 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:42.854205 11904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.855333 11882 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:42.855472 11882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "e6d7dffcff3140fa8f810f23dba66cc3"
format_stamp: "Formatted at 2026-08-06 00:44:42 on dist-test-slave-n326"
server_key: "91260bb2d098229f53c40244d55a20b9"
server_key_iv: "165a3f2f336a2854cbeef11c8f37fc5f"
server_key_version: "encryptionkey@0"
I20260806 00:44:42.855568 11882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:42.875989 11882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:42.876570 11882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:42.876746 11882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:42.877256 11882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:42.878079 11882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:42.878125 11882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.878186 11882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:42.878221 11882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:42.886607 11882 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:45501
I20260806 00:44:42.886663 12079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:45501 every 8 connection(s)
I20260806 00:44:42.887485 11882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:42.888913 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 11882
I20260806 00:44:42.889029 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:42.889321 30034 external_mini_cluster.cc:1428] Setting key bb0c2198fab208b579ee286eff700a93
I20260806 00:44:42.896382 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:42.889119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:40117 (local address 127.29.84.190:42553)
0806 00:44:42.889256 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:42.889260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:42.890499 (+ 1239us) server_negotiation.cc:407] Connection header received
0806 00:44:42.892043 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:42.892047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:42.892101 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:42.892204 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:42.893019 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.893555 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.894665 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.894897 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.895002 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:42.895926 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:42.895929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:42.895932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:42.895944 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:42.895964 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:42.895977 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:42.896046 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:42.896198 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 00:44:42.896253 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:44:42.897128 12083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:42.889594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:42553 (local address 127.29.84.131:40117)
0806 00:44:42.890370 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:42.890410 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:42.891844 (+ 1434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:42.892367 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:42.892373 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:42.892451 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:42.892872 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:42.892883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.893679 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.893682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:42.894540 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:42.894549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:42.895037 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:42.895041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:42.895219 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:42.895697 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:42.895698 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:42.895732 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:42.895787 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:42.896060 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:42.896071 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:42.896081 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 00:44:42.896193 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 00:44:42.896436 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":143,"threads_started":1}
I20260806 00:44:42.898095 12080 heartbeater.cc:347] Connected to a master server at 127.29.84.190:42553
I20260806 00:44:42.898311 12080 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:42.898841 12080 heartbeater.cc:511] Master 127.29.84.190:42553 requested a full tablet report, sending...
I20260806 00:44:42.899919 11398 ts_manager.cc:194] Registered new tserver with Master: e6d7dffcff3140fa8f810f23dba66cc3 (127.29.84.131:45501)
I20260806 00:44:42.900549 11398 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.131:40117
I20260806 00:44:42.903038 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 00:44:42.905619 11456 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:42.904883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.29.84.190:42553)
0806 00:44:42.904988 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:42.904991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:42.905182 (+ 191us) server_negotiation.cc:407] Connection header received
0806 00:44:42.905272 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:42.905274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:42.905308 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:42.905370 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:42.905532 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57662: BlockingRecv error: recv got EOF from 127.0.0.1:57662 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:44:42.905831 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 11484
I20260806 00:44:42.921751 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 11678
I20260806 00:44:42.929949 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 11882
I20260806 00:44:42.938414 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 11352
2026-08-06T00:44:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (841 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T00:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:42Z Disabled control of system clock
I20260806 00:44:42.969671 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35563
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:35563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 00:44:43.073477 12093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:43.073679 12093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:43.073714 12093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:43.077481 12093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:43.077527 12093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:43.077543 12093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:43.077576 12093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:43.077591 12093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:43.081967 12093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:35563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:43.082751 12093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:43.083885 12093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:43.091570 12107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:43.091579 12103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:43.091732 12093 server_base.cc:1034] running on GCE node
W20260806 00:44:43.091912 12100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:43.092466 12093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:43.093358 12093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:43.094513 12093 hybrid_clock.cc:648] HybridClock initialized: now 1785977083094496 us; error 29 us; skew 500 ppm
I20260806 00:44:43.096302 12093 webserver.cc:460] Webserver started at http://127.29.84.190:44289/ using document root <none> and password file <none>
I20260806 00:44:43.096810 12093 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:43.096870 12093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:43.097100 12093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:43.098958 12093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "bfd75aa888904e819c169da88a3e0d0f"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "779b720afa4fdb92caaa303eb27f311a"
server_key_iv: "5bec918cd29eafe1c67838b54869981c"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.099529 12093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "bfd75aa888904e819c169da88a3e0d0f"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "779b720afa4fdb92caaa303eb27f311a"
server_key_iv: "5bec918cd29eafe1c67838b54869981c"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.102870 12093 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:44:43.105119 12118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.106169 12093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:43.106288 12093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "bfd75aa888904e819c169da88a3e0d0f"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "779b720afa4fdb92caaa303eb27f311a"
server_key_iv: "5bec918cd29eafe1c67838b54869981c"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.106377 12093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:43.110509 12093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:43.111037 12093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:43.111253 12093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:43.117491 12093 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:35563
I20260806 00:44:43.117533 12202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:35563 every 8 connection(s)
I20260806 00:44:43.118305 12093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:43.121304 12203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:43.124907 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 12093
I20260806 00:44:43.125037 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:43.125320 30034 external_mini_cluster.cc:1428] Setting key 5db15820d065f1b8e0801a1498551b30
I20260806 00:44:43.126807 12203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f: Bootstrap starting.
I20260806 00:44:43.129230 12203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:43.130069 12203 log.cc:824] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:43.130754 12208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:43.126566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52170 (local address 127.29.84.190:35563)
0806 00:44:43.127217 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:43.127235 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:43.127272 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:44:43.128077 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:43.128098 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:43.128477 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:43.128797 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:43.129469 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:43.129476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:43.129492 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:43.129506 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:43.129566 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:43.129586 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:43.129691 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:43.129758 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 00:44:43.129982 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":103,"threads_started":1}
I20260806 00:44:43.132098 12203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f: No bootstrap required, opened a new log
I20260806 00:44:43.135419 12203 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd75aa888904e819c169da88a3e0d0f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35563 } }
I20260806 00:44:43.135612 12203 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:43.135685 12203 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd75aa888904e819c169da88a3e0d0f, State: Initialized, Role: FOLLOWER
I20260806 00:44:43.136089 12203 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd75aa888904e819c169da88a3e0d0f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35563 } }
I20260806 00:44:43.136224 12203 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:43.136297 12203 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:43.136433 12203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:43.137229 12203 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd75aa888904e819c169da88a3e0d0f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35563 } }
I20260806 00:44:43.137545 12203 leader_election.cc:302] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfd75aa888904e819c169da88a3e0d0f; no voters:
I20260806 00:44:43.137826 12203 leader_election.cc:288] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:43.137909 12212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:43.138245 12212 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [term 1 LEADER]: Becoming Leader. State: Replica: bfd75aa888904e819c169da88a3e0d0f, State: Running, Role: LEADER
I20260806 00:44:43.138583 12212 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd75aa888904e819c169da88a3e0d0f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35563 } }
I20260806 00:44:43.139019 12203 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:43.140297 12215 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd75aa888904e819c169da88a3e0d0f. Latest consensus state: current_term: 1 leader_uuid: "bfd75aa888904e819c169da88a3e0d0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd75aa888904e819c169da88a3e0d0f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35563 } } }
I20260806 00:44:43.140348 12213 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfd75aa888904e819c169da88a3e0d0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd75aa888904e819c169da88a3e0d0f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35563 } } }
I20260806 00:44:43.140415 12215 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:43.140446 12213 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:43.140759 12228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:43.143671 12228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:43.148586 12228 catalog_manager.cc:1347] Generated new cluster ID: f2e262d9de91434aa3c18a8270d39a34
I20260806 00:44:43.148646 12228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:43.156915 12228 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:43.157196 12228 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:43.168367 12228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfd75aa888904e819c169da88a3e0d0f: Generated new TSK 0
I20260806 00:44:43.169029 12228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:44:43.171744 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35563
--builtin_ntp_servers=127.29.84.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 00:44:43.279671 12241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:43.279929 12241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:43.279999 12241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:43.283864 12241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:43.283932 12241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:43.284016 12241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:43.288551 12241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:43.289439 12241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:43.290681 12241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:43.300242 12248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:43.300253 12247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:43.300314 12241 server_base.cc:1034] running on GCE node
W20260806 00:44:43.300285 12250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:43.300872 12241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:43.301465 12241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:43.302651 12241 hybrid_clock.cc:648] HybridClock initialized: now 1785977083302639 us; error 31 us; skew 500 ppm
I20260806 00:44:43.304477 12241 webserver.cc:460] Webserver started at http://127.29.84.129:39187/ using document root <none> and password file <none>
I20260806 00:44:43.305028 12241 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:43.305124 12241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:43.305335 12241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:43.307320 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "ed2e20865e9149c78ac8f05f9145f577"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "e783661f138bb29f45f224e111523262"
server_key_iv: "73e9fc3655ee872d87e0d853cdc8f3a2"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.307866 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "ed2e20865e9149c78ac8f05f9145f577"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "e783661f138bb29f45f224e111523262"
server_key_iv: "73e9fc3655ee872d87e0d853cdc8f3a2"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.311528 12241 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:44:43.313879 12259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.314950 12241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:43.315124 12241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "ed2e20865e9149c78ac8f05f9145f577"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "e783661f138bb29f45f224e111523262"
server_key_iv: "73e9fc3655ee872d87e0d853cdc8f3a2"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.315235 12241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:43.319437 12241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:43.319989 12241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:43.320176 12241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:43.320677 12241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:43.321523 12241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:43.321591 12241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.321647 12241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:43.321694 12241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.330463 12241 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:39653
I20260806 00:44:43.330511 12438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:39653 every 8 connection(s)
I20260806 00:44:43.331316 12241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:43.336935 12208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:43.333016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:44717 (local address 127.29.84.190:35563)
0806 00:44:43.333144 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:43.333147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:43.334059 (+ 912us) server_negotiation.cc:407] Connection header received
0806 00:44:43.335305 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:43.335309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:43.335355 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:43.335439 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:43.336460 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:43.336464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:43.336467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:43.336479 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:43.336500 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:43.336515 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:43.336610 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:43.336750 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 00:44:43.336794 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:44:43.337668 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:43.333335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35563 (local address 127.29.84.129:44717)
0806 00:44:43.333920 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:43.333956 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:43.335109 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:43.335585 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:43.335591 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:43.335689 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:43.336191 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:43.336193 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:43.336230 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:43.336293 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:43.336595 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:43.336604 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:43.336609 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:44:43.336779 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 00:44:43.337018 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":99,"threads_started":1}
I20260806 00:44:43.338793 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 12241
I20260806 00:44:43.338793 12439 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35563
I20260806 00:44:43.338927 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:43.339006 12439 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:43.339289 30034 external_mini_cluster.cc:1428] Setting key cda94c3539a198b56fd80ecb3b781848
I20260806 00:44:43.339493 12439 heartbeater.cc:511] Master 127.29.84.190:35563 requested a full tablet report, sending...
I20260806 00:44:43.341070 12145 ts_manager.cc:194] Registered new tserver with Master: ed2e20865e9149c78ac8f05f9145f577 (127.29.84.129:39653)
I20260806 00:44:43.341298 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35563
--builtin_ntp_servers=127.29.84.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 00:44:43.449363 12444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:43.449573 12444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:43.449616 12444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:43.453508 12444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:43.453552 12444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:43.453644 12444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:43.458158 12444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:43.459022 12444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:43.460237 12444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:43.468856 12453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:43.468847 12451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:43.469014 12444 server_base.cc:1034] running on GCE node
W20260806 00:44:43.468847 12457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:43.469619 12444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:43.470176 12444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:43.471370 12444 hybrid_clock.cc:648] HybridClock initialized: now 1785977083471308 us; error 80 us; skew 500 ppm
I20260806 00:44:43.473057 12444 webserver.cc:460] Webserver started at http://127.29.84.130:46467/ using document root <none> and password file <none>
I20260806 00:44:43.473515 12444 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:43.473567 12444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:43.473713 12444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:43.475585 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "36f36c7eb0a64b08bbf134fbb665108f"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "ebd5ab0eed208c8042c37ed8400488c9"
server_key_iv: "d3421b3c1ba14c7219b0b8d5fc9697e8"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.476059 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "36f36c7eb0a64b08bbf134fbb665108f"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "ebd5ab0eed208c8042c37ed8400488c9"
server_key_iv: "d3421b3c1ba14c7219b0b8d5fc9697e8"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.479343 12444 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:44:43.481571 12465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.482698 12444 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:43.482816 12444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "36f36c7eb0a64b08bbf134fbb665108f"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "ebd5ab0eed208c8042c37ed8400488c9"
server_key_iv: "d3421b3c1ba14c7219b0b8d5fc9697e8"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.482913 12444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:43.486790 12444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:43.487381 12444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:43.487546 12444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:43.488025 12444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:43.488839 12444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:43.488902 12444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.488965 12444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:43.489009 12444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.497465 12444 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:43563
I20260806 00:44:43.497509 12642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:43563 every 8 connection(s)
I20260806 00:44:43.498283 12444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:43.503677 12208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:43.499985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:53773 (local address 127.29.84.190:35563)
0806 00:44:43.500108 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:43.500112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:43.501005 (+ 893us) server_negotiation.cc:407] Connection header received
0806 00:44:43.502204 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:43.502206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:43.502242 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:43.502307 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:43.503307 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:43.503310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:43.503312 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:43.503324 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:43.503362 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:43.503371 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:43.503444 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:43.503532 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 00:44:43.503568 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 00:44:43.504369 12645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:43.500266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35563 (local address 127.29.84.130:53773)
0806 00:44:43.500862 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:43.500906 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:43.502020 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:43.502449 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:43.502455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:43.502545 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:43.503010 (+ 465us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:43.503012 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:43.503046 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:43.503159 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:43.503430 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:43.503435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:43.503438 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:44:43.503551 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 00:44:43.503715 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":99,"threads_started":1}
I20260806 00:44:43.505424 12643 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35563
I20260806 00:44:43.505636 12643 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:43.506014 12643 heartbeater.cc:511] Master 127.29.84.190:35563 requested a full tablet report, sending...
I20260806 00:44:43.506961 12145 ts_manager.cc:194] Registered new tserver with Master: 36f36c7eb0a64b08bbf134fbb665108f (127.29.84.130:43563)
I20260806 00:44:43.507555 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 12444
I20260806 00:44:43.507696 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:43.507962 30034 external_mini_cluster.cc:1428] Setting key c1ff8124c70aa6aa68e954f26a2ea2e3
I20260806 00:44:43.509841 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35563
--builtin_ntp_servers=127.29.84.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 00:44:43.616833 12646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:43.617089 12646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:43.617163 12646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:43.621290 12646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:43.621344 12646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:43.621446 12646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:43.626111 12646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:43.627167 12646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:43.628537 12646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:43.638125 12660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:43.638125 12655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:43.638176 12646 server_base.cc:1034] running on GCE node
W20260806 00:44:43.638131 12656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:43.638741 12646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:43.639437 12646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:43.640621 12646 hybrid_clock.cc:648] HybridClock initialized: now 1785977083640596 us; error 36 us; skew 500 ppm
I20260806 00:44:43.642594 12646 webserver.cc:460] Webserver started at http://127.29.84.131:45247/ using document root <none> and password file <none>
I20260806 00:44:43.643224 12646 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:43.643327 12646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:43.643567 12646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:43.645681 12646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "70d98fc1bca84a44aee97547d1ee0704"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "1dd2994026cfa562555ab57872ef6715"
server_key_iv: "e8b78eb01b050e69b64b9538a6dcdf51"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.646270 12646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "70d98fc1bca84a44aee97547d1ee0704"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "1dd2994026cfa562555ab57872ef6715"
server_key_iv: "e8b78eb01b050e69b64b9538a6dcdf51"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.649915 12646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 00:44:43.652431 12672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.653632 12646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:44:43.653760 12646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "70d98fc1bca84a44aee97547d1ee0704"
format_stamp: "Formatted at 2026-08-06 00:44:43 on dist-test-slave-n326"
server_key: "1dd2994026cfa562555ab57872ef6715"
server_key_iv: "e8b78eb01b050e69b64b9538a6dcdf51"
server_key_version: "encryptionkey@0"
I20260806 00:44:43.653870 12646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:43.658268 12646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:43.658828 12646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:43.658996 12646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:43.659533 12646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:43.660434 12646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:43.660502 12646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.660567 12646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:43.660616 12646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:43.669508 12646 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:42853
I20260806 00:44:43.669533 12851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:42853 every 8 connection(s)
I20260806 00:44:43.670388 12646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:43.676206 12208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:43.672146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:37511 (local address 127.29.84.190:35563)
0806 00:44:43.672270 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:43.672279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:43.673273 (+ 994us) server_negotiation.cc:407] Connection header received
0806 00:44:43.674541 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:43.674543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:43.674582 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:43.674654 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:43.675827 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:43.675830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:43.675832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:44:43.675844 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:43.675864 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:44:43.675872 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:43.675956 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:43.676064 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 00:44:43.676101 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:44:43.676656 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 12646
I20260806 00:44:43.676761 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:43.677055 30034 external_mini_cluster.cc:1428] Setting key 37f8b36a0ce58f487f709f5258c54d3f
I20260806 00:44:43.677135 12854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:43.672469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35563 (local address 127.29.84.131:37511)
0806 00:44:43.673099 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:43.673139 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:43.674355 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:43.674810 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:43.674816 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:43.674940 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:43.675566 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:44:43.675568 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:43.675612 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:44:43.675680 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:43.675947 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:43.675954 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:43.675957 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:44:43.676084 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 00:44:43.676299 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":111,"threads_started":1}
I20260806 00:44:43.678293 12852 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35563
I20260806 00:44:43.678534 12852 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:43.678920 12852 heartbeater.cc:511] Master 127.29.84.190:35563 requested a full tablet report, sending...
I20260806 00:44:43.679926 12146 ts_manager.cc:194] Registered new tserver with Master: 70d98fc1bca84a44aee97547d1ee0704 (127.29.84.131:42853)
I20260806 00:44:43.690696 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 00:44:43.693411 12208 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:44:43.692657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.29.84.190:35563)
0806 00:44:43.692770 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:43.692774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:43.692932 (+ 158us) server_negotiation.cc:407] Connection header received
0806 00:44:43.693031 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:43.693033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:43.693063 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:43.693129 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:43.693318 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52176: BlockingRecv error: recv got EOF from 127.0.0.1:52176 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:44:43.693743 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 12241
I20260806 00:44:43.705590 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 12444
I20260806 00:44:43.723927 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 12646
I20260806 00:44:43.735492 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 12093
2026-08-06T00:44:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (798 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:43 dist-test-slave-n326 krb5kdc[12866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:43 dist-test-slave-n326 krb5kdc[12866](info): set up 2 sockets
Aug 06 00:44:43 dist-test-slave-n326 krb5kdc[12866](info): commencing operation
krb5kdc: starting...
W20260806 00:44:44.977569 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:45 dist-test-slave-n326 krb5kdc[12866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:44:45.092176 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:45937
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:38449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:45937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:45.196357 12886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:45.196591 12886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:45.196630 12886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:45.200573 12886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:44:45.200623 12886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:45.200642 12886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:45.200659 12886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:44:45.200675 12886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:44:45.205585 12886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:45937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:45937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:45.206459 12886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:45.207361 12886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:45.216253 12892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:45.216264 12896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:45.216390 12886 server_base.cc:1034] running on GCE node
W20260806 00:44:45.216599 12893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:45.217207 12886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:45.218199 12886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:45.219383 12886 hybrid_clock.cc:648] HybridClock initialized: now 1785977085219365 us; error 29 us; skew 500 ppm
Aug 06 00:44:45 dist-test-slave-n326 krb5kdc[12866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977085, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:45.225102 12886 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:44:45.225867 12886 webserver.cc:460] Webserver started at http://127.29.84.190:39687/ using document root <none> and password file <none>
I20260806 00:44:45.226353 12886 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:45.226410 12886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:45.226569 12886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:45.228510 12886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "c7380d263ab74b2b82d358a6b10de7d4"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "75b1321538b983a24c0ad1aa62f76606"
server_key_iv: "e3d7f6182dbf2e1caee4cf59e7b48d26"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.229040 12886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "c7380d263ab74b2b82d358a6b10de7d4"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "75b1321538b983a24c0ad1aa62f76606"
server_key_iv: "e3d7f6182dbf2e1caee4cf59e7b48d26"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.232501 12886 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:44:45.234681 12905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.235839 12886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:44:45.235960 12886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "c7380d263ab74b2b82d358a6b10de7d4"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "75b1321538b983a24c0ad1aa62f76606"
server_key_iv: "e3d7f6182dbf2e1caee4cf59e7b48d26"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.236053 12886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:45.261297 12886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:45.264430 12886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:45.264628 12886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:45.271384 12886 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:45937
I20260806 00:44:45.271699 12983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:45937 every 8 connection(s)
I20260806 00:44:45.272333 12886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:44:45.274922 12985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:45.277469 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 12886
I20260806 00:44:45.277601 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:44:45.277922 30034 external_mini_cluster.cc:1428] Setting key 5f9b183f1293a9886620fb8048dd4c2c
I20260806 00:44:45.280752 12985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4: Bootstrap starting.
I20260806 00:44:45.282766 12985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:45.283515 12985 log.cc:824] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:45.285449 12985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4: No bootstrap required, opened a new log
Aug 06 00:44:45 dist-test-slave-n326 krb5kdc[12866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:45.289017 12985 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7380d263ab74b2b82d358a6b10de7d4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 45937 } }
I20260806 00:44:45.289235 12985 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:45.289297 12985 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7380d263ab74b2b82d358a6b10de7d4, State: Initialized, Role: FOLLOWER
I20260806 00:44:45.289732 12985 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7380d263ab74b2b82d358a6b10de7d4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 45937 } }
I20260806 00:44:45.289891 12985 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:44:45.289960 12985 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:44:45.290079 12985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:45.291085 12985 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7380d263ab74b2b82d358a6b10de7d4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 45937 } }
I20260806 00:44:45.291429 12985 leader_election.cc:302] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7380d263ab74b2b82d358a6b10de7d4; no voters:
I20260806 00:44:45.291747 12985 leader_election.cc:288] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:44:45.291846 12992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:45.292044 12992 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [term 1 LEADER]: Becoming Leader. State: Replica: c7380d263ab74b2b82d358a6b10de7d4, State: Running, Role: LEADER
I20260806 00:44:45.292232 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:45.279325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.29.84.190:45937)
0806 00:44:45.279739 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:45.279753 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:45.279787 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:44:45.280371 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:45.280386 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:45.280690 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:45.281033 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:45.281836 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.282783 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.283735 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.284528 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.284710 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:45.287424 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:45.287429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:45.287459 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:45.287485 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:45.289123 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.289622 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.289627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.289633 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.289745 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.290062 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.290065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.290067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.290500 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:45.290658 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:45.290946 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 00:44:45.291255 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":114,"threads_started":1}
I20260806 00:44:45.292914 12985 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:44:45.293294 12992 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7380d263ab74b2b82d358a6b10de7d4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 45937 } }
I20260806 00:44:45.295008 12994 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7380d263ab74b2b82d358a6b10de7d4. Latest consensus state: current_term: 1 leader_uuid: "c7380d263ab74b2b82d358a6b10de7d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7380d263ab74b2b82d358a6b10de7d4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 45937 } } }
I20260806 00:44:45.295357 12994 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [sys.catalog]: This master's current role is: LEADER
I20260806 00:44:45.295303 12993 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7380d263ab74b2b82d358a6b10de7d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7380d263ab74b2b82d358a6b10de7d4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 45937 } } }
I20260806 00:44:45.295550 12993 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4 [sys.catalog]: This master's current role is: LEADER
W20260806 00:44:45.297788 13010 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:44:45.297874 13010 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:44:45.297956 13012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:44:45.298606 13012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:44:45.303571 13012 catalog_manager.cc:1347] Generated new cluster ID: 5df6df9b4064476ba75045cafc9268cd
I20260806 00:44:45.303632 13012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:44:45.310519 13012 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:44:45.311614 13012 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:44:45.321827 13012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4: Generated new TSK 0
I20260806 00:44:45.322499 13012 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:44:45.359941 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:45937
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:38449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:45.467180 13021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:45.467420 13021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:45.467492 13021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:45.471436 13021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:45.471501 13021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:45.471585 13021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:44:45.476522 13021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:45937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:45.477384 13021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:45.478262 13021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:45.487145 13027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:45.487146 13026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:45.487250 13021 server_base.cc:1034] running on GCE node
W20260806 00:44:45.487191 13030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:45.487720 13021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:45.488266 13021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:45.489424 13021 hybrid_clock.cc:648] HybridClock initialized: now 1785977085489410 us; error 23 us; skew 500 ppm
Aug 06 00:44:45 dist-test-slave-n326 krb5kdc[12866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977085, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:45.492446 13021 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:44:45.493280 13021 webserver.cc:460] Webserver started at http://127.29.84.129:41721/ using document root <none> and password file <none>
I20260806 00:44:45.493754 13021 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:45.493811 13021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:45.494014 13021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:45.496027 13021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "42c48f45268445378ca74fdcfd59fc16"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "453a97526b526f65aa364b6d2d09ea6f"
server_key_iv: "434158bd05f7da27f4e6e55935d82241"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.496510 13021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "42c48f45268445378ca74fdcfd59fc16"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "453a97526b526f65aa364b6d2d09ea6f"
server_key_iv: "434158bd05f7da27f4e6e55935d82241"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.499917 13021 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:44:45.502079 13043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.503219 13021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:44:45.503340 13021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "42c48f45268445378ca74fdcfd59fc16"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "453a97526b526f65aa364b6d2d09ea6f"
server_key_iv: "434158bd05f7da27f4e6e55935d82241"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.503440 13021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:45.536871 13021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:45.539875 13021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:45.540061 13021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:45.540565 13021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:45.541326 13021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:45.541389 13021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.541461 13021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:45.541508 13021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.549820 13021 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:34425
I20260806 00:44:45.549844 13208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:34425 every 8 connection(s)
I20260806 00:44:45.550658 13021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:44:45.555962 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 13021
I20260806 00:44:45.556085 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:44:45.556372 30034 external_mini_cluster.cc:1428] Setting key 6f10bd784178454f801c61470723c045
Aug 06 00:44:45 dist-test-slave-n326 krb5kdc[12866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977085, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:45.564859 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:45.552355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:49295 (local address 127.29.84.190:45937)
0806 00:44:45.552475 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:45.552480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:45.553229 (+ 749us) server_negotiation.cc:407] Connection header received
0806 00:44:45.554566 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:45.554570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:45.554630 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:45.554706 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:45.555807 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.556514 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.557798 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.558140 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.558271 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:45.560984 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:45.560991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:45.560994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:45.561028 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:45.562623 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.563112 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.563118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.563123 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.563231 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.563626 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.563632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.563635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.563867 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:45.563980 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:45.564458 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 00:44:45.564606 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:44:45.565351 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:45.552682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:45937 (local address 127.29.84.129:49295)
0806 00:44:45.553093 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:45.553131 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:45.554361 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:45.554857 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:45.554863 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:45.555234 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:45.555661 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:45.555674 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.556658 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.556662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:45.557664 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:45.557676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.558297 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.558299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:45.558449 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:45.559043 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:45.559090 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:45.560818 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:45.562732 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:45.562736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:45.562746 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:45.562981 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:45.563327 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:45.563330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:45.563331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:45.563517 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:45.563986 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:45.563991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:45.564195 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 00:44:45.564356 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 00:44:45.564574 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":101,"threads_started":1}
I20260806 00:44:45.566455 13209 heartbeater.cc:347] Connected to a master server at 127.29.84.190:45937
I20260806 00:44:45.566684 13209 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:45.567167 13209 heartbeater.cc:511] Master 127.29.84.190:45937 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:44:45.568655 12928 ts_manager.cc:194] Registered new tserver with Master: 42c48f45268445378ca74fdcfd59fc16 (127.29.84.129:34425)
I20260806 00:44:45.569861 12928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:49295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:44:45.585439 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:45937
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:38449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:45.692673 13217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:45.692899 13217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:45.692938 13217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:45.696904 13217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:45.696954 13217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:45.697008 13217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:44:45.701974 13217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:45937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:45.702782 13217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:45.703909 13217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:45.712929 13223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:45.713078 13217 server_base.cc:1034] running on GCE node
W20260806 00:44:45.712929 13224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:45.712967 13227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:45.713526 13217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:45.714180 13217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:45.715353 13217 hybrid_clock.cc:648] HybridClock initialized: now 1785977085715337 us; error 29 us; skew 500 ppm
Aug 06 00:44:45 dist-test-slave-n326 krb5kdc[12866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977085, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:45.718152 13217 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:44:45.718930 13217 webserver.cc:460] Webserver started at http://127.29.84.130:34629/ using document root <none> and password file <none>
I20260806 00:44:45.719497 13217 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:45.719558 13217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:45.719763 13217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:45.721604 13217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "7e2a66cad8f6457b8d8e51c997c1abe1"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "ef337fd50a630bb3f0ce4b6795dc56e5"
server_key_iv: "fdbb6fe7f53f9b7a693fa5ebd604ee06"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.722138 13217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "7e2a66cad8f6457b8d8e51c997c1abe1"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "ef337fd50a630bb3f0ce4b6795dc56e5"
server_key_iv: "fdbb6fe7f53f9b7a693fa5ebd604ee06"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.725634 13217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 00:44:45.727895 13238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.729019 13217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:44:45.729147 13217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "7e2a66cad8f6457b8d8e51c997c1abe1"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "ef337fd50a630bb3f0ce4b6795dc56e5"
server_key_iv: "fdbb6fe7f53f9b7a693fa5ebd604ee06"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.729259 13217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:45.748839 13217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:45.751863 13217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:45.752045 13217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:45.752574 13217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:45.753319 13217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:45.753363 13217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.753398 13217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:45.753448 13217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.761969 13217 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:35749
I20260806 00:44:45.762007 13407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:35749 every 8 connection(s)
I20260806 00:44:45.762776 13217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:44:45.771196 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 13217
I20260806 00:44:45.771324 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:44:45.771607 30034 external_mini_cluster.cc:1428] Setting key c51955ff20492199dae4614dbff67ccf
Aug 06 00:44:45 dist-test-slave-n326 krb5kdc[12866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977085, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:45.776157 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:45.764421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:54291 (local address 127.29.84.190:45937)
0806 00:44:45.764569 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:45.764573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:45.765256 (+ 683us) server_negotiation.cc:407] Connection header received
0806 00:44:45.766573 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:45.766576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:45.766636 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:45.766731 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:45.767790 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.768301 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.769394 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.769603 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.769722 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:45.772296 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:45.772314 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:45.772316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:45.772339 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:45.773977 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.774468 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.774474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.774479 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.774590 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.774948 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.774953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.774957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.775221 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:45.775324 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:45.775954 (+ 630us) server_negotiation.cc:299] Negotiation successful
0806 00:44:45.776047 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:44:45.777246 13412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:45.764710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:45937 (local address 127.29.84.130:54291)
0806 00:44:45.765110 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:45.765149 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:45.766350 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:45.766893 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:45.766899 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:45.767236 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:45.767649 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:45.767661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.768435 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.768438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:45.769263 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:45.769272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.769758 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.769762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:45.769918 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:45.770496 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:45.770511 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:45.772118 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:45.774099 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:45.774104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:45.774114 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:45.774358 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:45.774702 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:45.774704 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:45.774706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:45.774860 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:45.775335 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:45.775344 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:45.775671 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 00:44:45.775892 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 00:44:45.776232 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":97,"threads_started":1}
I20260806 00:44:45.778342 13408 heartbeater.cc:347] Connected to a master server at 127.29.84.190:45937
I20260806 00:44:45.778577 13408 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:45.779006 13408 heartbeater.cc:511] Master 127.29.84.190:45937 requested a full tablet report, sending...
I20260806 00:44:45.780054 12928 ts_manager.cc:194] Registered new tserver with Master: 7e2a66cad8f6457b8d8e51c997c1abe1 (127.29.84.130:35749)
I20260806 00:44:45.780925 12928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:54291
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:44:45.801604 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:45937
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:38449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:44:45.910949 13418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:44:45.911218 13418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:44:45.911262 13418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:44:45.914781 13418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:44:45.914824 13418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:44:45.914875 13418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:44:45.919723 13418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:45937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:44:45.920552 13418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:44:45.921397 13418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:44:45.930312 13426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:45.930335 13432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:44:45.930330 13430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:44:45.930449 13418 server_base.cc:1034] running on GCE node
I20260806 00:44:45.930960 13418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:44:45.931541 13418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:44:45.932719 13418 hybrid_clock.cc:648] HybridClock initialized: now 1785977085932701 us; error 31 us; skew 500 ppm
Aug 06 00:44:45 dist-test-slave-n326 krb5kdc[12866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977085, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:44:45.935289 13418 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:44:45.936053 13418 webserver.cc:460] Webserver started at http://127.29.84.131:46033/ using document root <none> and password file <none>
I20260806 00:44:45.936555 13418 fs_manager.cc:359] Metadata directory not provided
I20260806 00:44:45.936640 13418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:44:45.936879 13418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:44:45.938710 13418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "9f2a1bb69d2d420984220276c7631387"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "afba179b92e4bad903f51523da96bd6f"
server_key_iv: "5b95d91709a48bed6aab156bf04d4515"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.939361 13418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "9f2a1bb69d2d420984220276c7631387"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "afba179b92e4bad903f51523da96bd6f"
server_key_iv: "5b95d91709a48bed6aab156bf04d4515"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.942847 13418 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 00:44:45.945060 13450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.946168 13418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:44:45.946298 13418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "9f2a1bb69d2d420984220276c7631387"
format_stamp: "Formatted at 2026-08-06 00:44:45 on dist-test-slave-n326"
server_key: "afba179b92e4bad903f51523da96bd6f"
server_key_iv: "5b95d91709a48bed6aab156bf04d4515"
server_key_version: "encryptionkey@0"
I20260806 00:44:45.946398 13418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:44:45.960947 13418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:44:45.963521 13418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:44:45.963698 13418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:44:45.964246 13418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:44:45.965013 13418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:44:45.965092 13418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.965155 13418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:44:45.965201 13418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:44:45.973414 13418 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:39279
I20260806 00:44:45.973439 13616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:39279 every 8 connection(s)
I20260806 00:44:45.974177 13418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:44:45.977334 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 13418
I20260806 00:44:45.977491 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:44:45.977847 30034 external_mini_cluster.cc:1428] Setting key 85903db1b8ce90f329df3f09f0bc9745
Aug 06 00:44:45 dist-test-slave-n326 krb5kdc[12866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977085, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:44:45.987491 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:45.975947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:45833 (local address 127.29.84.190:45937)
0806 00:44:45.976075 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:45.976080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:45.976841 (+ 761us) server_negotiation.cc:407] Connection header received
0806 00:44:45.978517 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:45.978520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:45.978577 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:45.978673 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:45.979844 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.980381 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.981512 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.981733 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.981836 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:45.984169 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:45.984173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:45.984175 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:45.984197 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:45.985588 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.986073 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.986077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.986079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.986160 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.986477 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.986482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.986483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.986652 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:45.986754 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:45.987215 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 00:44:45.987330 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:44:45.988152 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:45.976295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:45937 (local address 127.29.84.131:45833)
0806 00:44:45.976707 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:45.976746 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:45.978293 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:45.978852 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:45.978860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:45.979287 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:44:45.979709 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:45.979722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.980529 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.980533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:45.981390 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:45.981399 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.981884 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.981889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:45.982041 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:45.982564 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:44:45.982580 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:44:45.984008 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:44:45.985719 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:45.985724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:45.985737 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:45.985974 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:45.986249 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:44:45.986253 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:44:45.986255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:44:45.986385 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:44:45.986753 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:44:45.986760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:44:45.986977 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 00:44:45.987174 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 00:44:45.987454 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
I20260806 00:44:45.989075 13617 heartbeater.cc:347] Connected to a master server at 127.29.84.190:45937
I20260806 00:44:45.989264 13617 heartbeater.cc:464] Registering TS with master...
I20260806 00:44:45.989642 13617 heartbeater.cc:511] Master 127.29.84.190:45937 requested a full tablet report, sending...
I20260806 00:44:45.990590 12928 ts_manager.cc:194] Registered new tserver with Master: 9f2a1bb69d2d420984220276c7631387 (127.29.84.131:39279)
I20260806 00:44:45.991214 12928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:45833
I20260806 00:44:45.991510 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:44:45.999716 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:45.993332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.29.84.190:45937)
0806 00:44:45.993416 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:45.993419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:45.993611 (+ 192us) server_negotiation.cc:407] Connection header received
0806 00:44:45.993693 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:45.993694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:45.993725 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:45.993784 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:44:45.994403 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.994821 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.995649 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:45.995846 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:45.995932 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:45.997187 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:44:45.997190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:44:45.997192 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:44:45.997215 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:44:45.998338 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.998707 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.998709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.998710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.998781 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:44:45.999006 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:44:45.999008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:44:45.999009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:44:45.999262 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:44:45.999365 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:45.999535 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 00:44:45.999618 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 00:44:46.002141 12928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:44:46.005079 12928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:44:46.018925 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.013185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:39279 (local address 127.0.0.1:49822)
0806 00:44:46.013481 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:46.013495 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:46.013623 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:46.014863 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:46.014869 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:46.014907 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:46.015014 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.015020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.016454 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.016461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.017521 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.017532 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.018297 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.018300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.018502 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.018548 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 00:44:46.018654 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 00:44:46.018766 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":4,"spinlock_wait_cycles":5760,"thread_start_us":83,"threads_started":1}
I20260806 00:44:46.018980 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.014064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49822 (local address 127.29.84.131:39279)
0806 00:44:46.014476 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:46.014486 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:46.014503 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:44:46.014563 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:46.014568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:46.014716 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:46.015182 (+ 466us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:46.015314 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.016311 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.017642 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.018171 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.018285 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.018417 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:46.018723 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 00:44:46.018822 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":154,"threads_started":1}
I20260806 00:44:46.019527 13638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.013527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:35749 (local address 127.0.0.1:53610)
0806 00:44:46.013835 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:46.013847 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:46.013929 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:46.014680 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:46.014686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:46.014707 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:46.014835 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.014842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.016285 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.016288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.017607 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.017618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.018980 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.018984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.019198 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.019234 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 00:44:46.019309 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:44:46.019402 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":226,"thread_start_us":90,"threads_started":1}
I20260806 00:44:46.019565 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.013030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.29.84.129:34425)
0806 00:44:46.013709 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:46.013733 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:46.014282 (+ 549us) server_negotiation.cc:407] Connection header received
0806 00:44:46.014371 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:46.014378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:46.014580 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:46.014729 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:46.015120 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.015822 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.017397 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.018207 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.018308 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.018517 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:46.019368 (+ 851us) server_negotiation.cc:299] Negotiation successful
0806 00:44:46.019461 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":95,"threads_started":1}
I20260806 00:44:46.018925 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.012898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:34425 (local address 127.0.0.1:33402)
0806 00:44:46.013401 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:46.013449 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:46.014082 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:46.014843 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:46.014846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:46.014863 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:46.014977 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.014981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.015946 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.015951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.017272 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.017282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.018518 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.018521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.018638 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.018666 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:44:46.018726 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 00:44:46.018780 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":46,"spinlock_wait_cycles":6400,"thread_start_us":54,"threads_started":1}
I20260806 00:44:46.020004 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.013830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53610 (local address 127.29.84.130:35749)
0806 00:44:46.014290 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:46.014300 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:46.014319 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:44:46.014378 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:46.014381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:46.014523 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:46.014678 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:46.014946 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.016125 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.017736 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.018677 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.018783 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.018905 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:46.019745 (+ 840us) server_negotiation.cc:299] Negotiation successful
0806 00:44:46.019845 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":117,"threads_started":1}
I20260806 00:44:46.022205 13115 tablet_service.cc:1467] Processing CreateTablet for tablet 6d8db442cd3842f692d5a8800897f15d (DEFAULT_TABLE table=test-table [id=013024a6af2643eba555484346e92bcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:46.022212 13528 tablet_service.cc:1467] Processing CreateTablet for tablet 6d8db442cd3842f692d5a8800897f15d (DEFAULT_TABLE table=test-table [id=013024a6af2643eba555484346e92bcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:46.022490 13317 tablet_service.cc:1467] Processing CreateTablet for tablet 6d8db442cd3842f692d5a8800897f15d (DEFAULT_TABLE table=test-table [id=013024a6af2643eba555484346e92bcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:44:46.023267 13115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8db442cd3842f692d5a8800897f15d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:46.023288 13528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8db442cd3842f692d5a8800897f15d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:46.023396 13317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8db442cd3842f692d5a8800897f15d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:44:46.029819 13641 tablet_bootstrap.cc:492] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16: Bootstrap starting.
I20260806 00:44:46.030578 13642 tablet_bootstrap.cc:492] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: Bootstrap starting.
I20260806 00:44:46.030965 13644 tablet_bootstrap.cc:492] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: Bootstrap starting.
I20260806 00:44:46.031898 13641 tablet_bootstrap.cc:654] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:46.032971 13644 tablet_bootstrap.cc:654] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:46.032982 13641 log.cc:824] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:46.033346 13642 tablet_bootstrap.cc:654] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: Neither blocks nor log segments found. Creating new log.
I20260806 00:44:46.034235 13642 log.cc:824] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:46.034235 13644 log.cc:824] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: Log is configured to *not* fsync() on all Append() calls
I20260806 00:44:46.034869 13641 tablet_bootstrap.cc:492] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16: No bootstrap required, opened a new log
I20260806 00:44:46.035038 13641 ts_tablet_manager.cc:1397] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:44:46.036139 13644 tablet_bootstrap.cc:492] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: No bootstrap required, opened a new log
I20260806 00:44:46.036136 13642 tablet_bootstrap.cc:492] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: No bootstrap required, opened a new log
I20260806 00:44:46.036315 13644 ts_tablet_manager.cc:1397] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:44:46.036315 13642 ts_tablet_manager.cc:1397] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:44:46.038089 13641 raft_consensus.cc:348] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } }
I20260806 00:44:46.038321 13641 raft_consensus.cc:374] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:46.038371 13641 raft_consensus.cc:729] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c48f45268445378ca74fdcfd59fc16, State: Initialized, Role: FOLLOWER
I20260806 00:44:46.038808 13641 consensus_queue.cc:260] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } }
I20260806 00:44:46.039443 13209 heartbeater.cc:503] Master 127.29.84.190:45937 was elected leader, sending a full tablet report...
I20260806 00:44:46.040036 13644 raft_consensus.cc:348] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } }
I20260806 00:44:46.040043 13642 raft_consensus.cc:348] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } }
I20260806 00:44:46.040292 13644 raft_consensus.cc:374] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:46.040292 13642 raft_consensus.cc:374] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:44:46.040359 13642 raft_consensus.cc:729] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2a66cad8f6457b8d8e51c997c1abe1, State: Initialized, Role: FOLLOWER
I20260806 00:44:46.040350 13644 raft_consensus.cc:729] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2a1bb69d2d420984220276c7631387, State: Initialized, Role: FOLLOWER
I20260806 00:44:46.040759 13642 consensus_queue.cc:260] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } }
I20260806 00:44:46.040759 13644 consensus_queue.cc:260] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } }
I20260806 00:44:46.041085 13641 ts_tablet_manager.cc:1428] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260806 00:44:46.041709 13642 ts_tablet_manager.cc:1428] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:44:46.041709 13644 ts_tablet_manager.cc:1428] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:44:46.042024 13617 heartbeater.cc:503] Master 127.29.84.190:45937 was elected leader, sending a full tablet report...
I20260806 00:44:46.042049 13408 heartbeater.cc:503] Master 127.29.84.190:45937 was elected leader, sending a full tablet report...
W20260806 00:44:46.051735 13210 tablet.cc:2369] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:44:46.070791 13655 raft_consensus.cc:482] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:44:46.070946 13655 raft_consensus.cc:504] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } }
I20260806 00:44:46.071938 13655 leader_election.cc:288] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42c48f45268445378ca74fdcfd59fc16 (127.29.84.129:34425), 7e2a66cad8f6457b8d8e51c997c1abe1 (127.29.84.130:35749)
I20260806 00:44:46.075865 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.072169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:56083 (local address 127.29.84.129:34425)
0806 00:44:46.072280 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:46.072285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:46.072495 (+ 210us) server_negotiation.cc:407] Connection header received
0806 00:44:46.072544 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:46.072547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:46.072596 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:46.072685 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:46.072992 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.073633 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.074868 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.075284 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.075398 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.075448 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:46.075663 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 00:44:46.075735 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:44:46.075882 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.072294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:60637 (local address 127.29.84.130:35749)
0806 00:44:46.072406 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:46.072411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:46.072697 (+ 286us) server_negotiation.cc:407] Connection header received
0806 00:44:46.072786 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:46.072789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:46.072832 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:46.072918 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:44:46.073181 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.073709 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.074846 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.075292 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.075414 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.075464 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:46.075693 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 00:44:46.075764 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:44:46.075901 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.072112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:34425 (local address 127.29.84.131:56083)
0806 00:44:46.072301 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:46.072317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:46.072405 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:46.072720 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:46.072725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:46.072750 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:46.072876 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.072886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.073744 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.073750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.074722 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.074732 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.075397 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.075400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.075532 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.075557 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:44:46.075619 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 00:44:46.075741 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":71}
I20260806 00:44:46.075930 13658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.072162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:35749 (local address 127.29.84.131:60637)
0806 00:44:46.072570 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:46.072583 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:44:46.072699 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:44:46.072931 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:44:46.072949 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:44:46.072969 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:44:46.073076 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.073081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.073798 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.073801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.074716 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:44:46.074726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.075418 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.075421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:44:46.075546 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.075564 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:44:46.075661 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:44:46.075742 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":40,"thread_start_us":111,"threads_started":1}
I20260806 00:44:46.076531 13144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8db442cd3842f692d5a8800897f15d" candidate_uuid: "9f2a1bb69d2d420984220276c7631387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42c48f45268445378ca74fdcfd59fc16" is_pre_election: true
I20260806 00:44:46.076651 13349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8db442cd3842f692d5a8800897f15d" candidate_uuid: "9f2a1bb69d2d420984220276c7631387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" is_pre_election: true
I20260806 00:44:46.076800 13144 raft_consensus.cc:2454] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2a1bb69d2d420984220276c7631387 in term 0.
I20260806 00:44:46.076943 13349 raft_consensus.cc:2454] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2a1bb69d2d420984220276c7631387 in term 0.
I20260806 00:44:46.077275 13455 leader_election.cc:302] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42c48f45268445378ca74fdcfd59fc16, 9f2a1bb69d2d420984220276c7631387; no voters:
I20260806 00:44:46.077508 13655 raft_consensus.cc:2781] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:44:46.077590 13655 raft_consensus.cc:482] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:44:46.077633 13655 raft_consensus.cc:3037] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:46.078554 13655 raft_consensus.cc:504] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } }
I20260806 00:44:46.078876 13655 leader_election.cc:288] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [CANDIDATE]: Term 1 election: Requested vote from peers 42c48f45268445378ca74fdcfd59fc16 (127.29.84.129:34425), 7e2a66cad8f6457b8d8e51c997c1abe1 (127.29.84.130:35749)
I20260806 00:44:46.079195 13144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8db442cd3842f692d5a8800897f15d" candidate_uuid: "9f2a1bb69d2d420984220276c7631387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42c48f45268445378ca74fdcfd59fc16"
I20260806 00:44:46.079275 13349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8db442cd3842f692d5a8800897f15d" candidate_uuid: "9f2a1bb69d2d420984220276c7631387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1"
I20260806 00:44:46.079315 13144 raft_consensus.cc:3037] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:46.079389 13349 raft_consensus.cc:3037] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:44:46.080577 13349 raft_consensus.cc:2454] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2a1bb69d2d420984220276c7631387 in term 1.
I20260806 00:44:46.080577 13144 raft_consensus.cc:2454] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2a1bb69d2d420984220276c7631387 in term 1.
I20260806 00:44:46.080840 13455 leader_election.cc:302] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42c48f45268445378ca74fdcfd59fc16, 9f2a1bb69d2d420984220276c7631387; no voters:
I20260806 00:44:46.081007 13655 raft_consensus.cc:2781] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:44:46.081183 13655 raft_consensus.cc:686] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 1 LEADER]: Becoming Leader. State: Replica: 9f2a1bb69d2d420984220276c7631387, State: Running, Role: LEADER
I20260806 00:44:46.081468 13655 consensus_queue.cc:237] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } }
I20260806 00:44:46.084288 12926 catalog_manager.cc:5495] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f2a1bb69d2d420984220276c7631387 (127.29.84.131). New cstate: current_term: 1 leader_uuid: "9f2a1bb69d2d420984220276c7631387" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f2a1bb69d2d420984220276c7631387" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20260806 00:44:46.097038 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.093652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49838 (local address 127.29.84.131:39279)
0806 00:44:46.093756 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 00:44:46.093760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:44:46.093814 (+ 54us) server_negotiation.cc:407] Connection header received
0806 00:44:46.093959 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:44:46.093961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:44:46.094004 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:44:46.094076 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:44:46.094331 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.094792 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.095680 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:44:46.095900 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:44:46.095998 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:44:46.096303 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:44:46.096642 (+ 339us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:44:46.096767 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 00:44:46.096828 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 00:44:46.096897 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 00:44:46.101933 13144 raft_consensus.cc:1261] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16 [term 1 FOLLOWER]: Refusing update from remote peer 9f2a1bb69d2d420984220276c7631387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:46.101933 13349 raft_consensus.cc:1261] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [term 1 FOLLOWER]: Refusing update from remote peer 9f2a1bb69d2d420984220276c7631387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:44:46.102607 13660 consensus_queue.cc:1035] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2a66cad8f6457b8d8e51c997c1abe1" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:46.102757 13655 consensus_queue.cc:1035] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42c48f45268445378ca74fdcfd59fc16" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:44:46.108995 13663 mvcc.cc:205] Tried to move back new op lower bound from 7315362144667742208 to 7315362144592248832. Current Snapshot: MvccSnapshot[applied={T|T < 7315362144667742208}]
I20260806 00:44:46.117132 12924 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:46.117281 12924 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:44:46.119653 12924 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7380d263ab74b2b82d358a6b10de7d4: Sending DeleteTablet for 3 replicas of tablet 6d8db442cd3842f692d5a8800897f15d
I20260806 00:44:46.120435 13115 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d8db442cd3842f692d5a8800897f15d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:46 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:33402
I20260806 00:44:46.120445 13317 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d8db442cd3842f692d5a8800897f15d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:46 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53610
I20260806 00:44:46.120620 13528 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d8db442cd3842f692d5a8800897f15d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:44:46 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:49822
I20260806 00:44:46.120886 13680 tablet_replica.cc:330] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: stopping tablet replica
I20260806 00:44:46.121155 13680 raft_consensus.cc:2229] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:44:46.121306 13679 tablet_replica.cc:330] T 6d8db442cd3842f692d5a8800897f15d P 42c48f45268445378ca74fdcfd59fc16: stopping tablet replica
I20260806 00:44:46.121444 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 13021
I20260806 00:44:46.144102 13680 raft_consensus.cc:2258] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:46.146832 13680 ts_tablet_manager.cc:1905] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:46.150875 13680 ts_tablet_manager.cc:1918] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:46.151034 13680 log.cc:1196] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/6d8db442cd3842f692d5a8800897f15d
I20260806 00:44:46.151511 13680 ts_tablet_manager.cc:1939] T 6d8db442cd3842f692d5a8800897f15d P 7e2a66cad8f6457b8d8e51c997c1abe1: Deleting consensus metadata
I20260806 00:44:46.153280 13681 tablet_replica.cc:330] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: stopping tablet replica
I20260806 00:44:46.153698 13681 raft_consensus.cc:2229] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:44:46.154182 13681 raft_consensus.cc:2258] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:44:46.156261 13681 ts_tablet_manager.cc:1905] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:44:46.160192 13681 ts_tablet_manager.cc:1918] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:44:46.160337 13681 log.cc:1196] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/6d8db442cd3842f692d5a8800897f15d
I20260806 00:44:46.160693 13681 ts_tablet_manager.cc:1939] T 6d8db442cd3842f692d5a8800897f15d P 9f2a1bb69d2d420984220276c7631387: Deleting consensus metadata
I20260806 00:44:46.161535 12907 catalog_manager.cc:4841] TS 7e2a66cad8f6457b8d8e51c997c1abe1 (127.29.84.130:35749): tablet 6d8db442cd3842f692d5a8800897f15d (table test-table [id=013024a6af2643eba555484346e92bcf]) successfully deleted
I20260806 00:44:46.162015 12907 catalog_manager.cc:4841] TS 9f2a1bb69d2d420984220276c7631387 (127.29.84.131:39279): tablet 6d8db442cd3842f692d5a8800897f15d (table test-table [id=013024a6af2643eba555484346e92bcf]) successfully deleted
W20260806 00:44:46.168431 12910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:34425 (error 108)
I20260806 00:44:46.168702 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 13217
I20260806 00:44:46.171566 13638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:44:46.171112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:34425 (local address 127.0.0.1:33416)
0806 00:44:46.171356 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 00:44:46.171454 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260806 00:44:46.171787 12910 catalog_manager.cc:4584] TS 42c48f45268445378ca74fdcfd59fc16 (127.29.84.129:34425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d8db442cd3842f692d5a8800897f15d: Network error: Client connection negotiation failed: client connection to 127.29.84.129:34425: connect: Connection refused (error 111)
I20260806 00:44:46.188155 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 13418
I20260806 00:44:46.195694 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 12886
2026-08-06T00:44:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2465 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:44:46 dist-test-slave-n326 krb5kdc[13687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:44:46 dist-test-slave-n326 krb5kdc[13687](info): set up 2 sockets
Aug 06 00:44:46 dist-test-slave-n326 krb5kdc[13687](info): commencing operation
krb5kdc: starting...
W20260806 00:44:47.402411 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:44:47 dist-test-slave-n326 krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:44:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.84.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.84.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/rangeradmin_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/rangeradmin_127.29.84.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.84.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.84.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/rangerlookup_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/rangerlookup_127.29.84.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.84.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.84.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/HTTP_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/HTTP_127.29.84.148@KRBTEST.COM.keytab.
I20260806 00:44:47.559279 30034 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskdVRgL9/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/postgres -l logfile start
2026-08-06 00:44:49.981 UTC [13722] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 00:44:49.981 UTC [13722] LOG: listening on IPv4 address "127.29.84.148", port 60899
2026-08-06 00:44:49.985 UTC [13722] LOG: listening on Unix socket "/tmp/.s.PGSQL.60899"
2026-08-06 00:44:50.019 UTC [13727] LOG: database system was shut down at 2026-08-06 00:44:48 UTC
2026-08-06 00:44:50.024 UTC [13722] LOG: database system is ready to accept connections
I20260806 00:44:51.120695 30034 mini_postgres.cc:96] Postgres bound to 60899
2026-08-06 00:44:51.128 UTC [13735] FATAL: database "slave" does not exist
127.29.84.148:60899 - accepting connections
I20260806 00:44:51.129238 30034 mini_ranger.cc:161] Starting Ranger...
I20260806 00:44:51.144333 30034 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 00:44:51.605974 30034 mini_ranger.cc:87] Created ranger Postgres database
I20260806 00:44:51.606096 30034 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-admin
2026-08-06 00:44:51,909 [I] DB FLAVOR :POSTGRES
2026-08-06 00:44:51,909 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:44:51,909 [I] Checking connection..
2026-08-06 00:44:51,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 00:44:52,166 [I] Checking connection passed.
2026-08-06 00:44:52,166 [I] --------- Verifying version history table ---------
2026-08-06 00:44:52,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:44:52,421 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:44:52,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:44:52,658 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:44:52,658 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 00:44:52,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 00:44:52.903 UTC [13828] WARNING: there is no transaction in progress
2026-08-06 00:44:52,909 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 00:44:52,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:44:53,152 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 00:44:53,152 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:44:53,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 00:44:53,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 00:44:53,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-n326.c.gcp-upstream.internal','N') ;"
2026-08-06 00:44:53,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:44:54,111 [I] Table x_portal_user does not exist in database ranger
2026-08-06 00:44:54,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:44:54,350 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 00:44:54,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:44:54,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:44:54,819 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 00:44:54,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 00:44:55.725 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.742 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.757 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.771 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.786 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.805 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.865 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.871 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.885 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.894 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.904 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.917 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.925 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.931 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.938 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:55.945 UTC [13987] WARNING: there is no transaction in progress
2026-08-06 00:44:56,538 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 00:44:56,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 00:44:56,762 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 00:44:56,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:44:57,017 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 00:44:57,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:44:57,262 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 00:44:57,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:44:57,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:44:57,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-n326.c.gcp-upstream.internal';"
2026-08-06 00:44:57,999 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 00:44:57,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 00:44:58,249 [I] DB_PATCHES have already been applied
I20260806 00:44:58.256220 30034 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-admin:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskdVRgL9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 00:44:58.256278 30034 mini_ranger.cc:193] Using host: 127.29.84.148
I20260806 00:44:58.260401 30034 mini_ranger.cc:227] Ranger admin URL: http://127.29.84.148:35985
Aug 06, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:44:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:44:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 00:44:59.549134 30034 mini_ranger.cc:160] Time spent starting Ranger: real 8.420s user 0.010s sys 0.002s
Aug 06, 2026 12:45:19 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:45:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 12:45:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:45:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:45:20 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 00:45:21.394371 30034 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.84.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.84.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/rangerkms_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/rangerkms_127.29.84.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.84.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.84.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.84.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/HTTP_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.84.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/HTTP_127.29.84.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 00:45:21 dist-test-slave-n326 krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977121, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 00:45:21.480129 30034 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 00:45:21.489720 30034 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 00:45:22.333272 30034 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 00:45:22.333376 30034 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-kms
I20260806 00:45:22.333482 30034 mini_ranger_kms.cc:227] Using postgres at 127.29.84.148:60899
2026-08-06 00:45:22,581 [I] DB FLAVOR :POSTGRES
2026-08-06 00:45:22,581 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:45:22,581 [I] Checking connection
2026-08-06 00:45:22,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 00:45:22,819 [I] connection success
2026-08-06 00:45:22,819 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 00:45:22,819 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 00:45:22,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 00:45:23,045 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 00:45:23,046 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:45:23,046 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 00:45:23,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:60899/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 00:45:23,294 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 00:45:23.759616 30034 mini_ranger_kms.cc:310] Created kms service
I20260806 00:45:23.954869 30034 mini_ranger_kms.cc:326] Created kudu user
I20260806 00:45:23.996018 30034 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 00:45:23 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977121, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:45:24.392711 30034 mini_ranger_kms.cc:384] Added ranger policy
I20260806 00:45:24.392865 30034 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-kms:/tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 00:45:24.392886 30034 mini_ranger_kms.cc:240] Using host: 127.29.84.148
I20260806 00:45:24.397496 30034 mini_ranger_kms.cc:276] Ranger KMS PID: 14276
I20260806 00:45:24.397629 30034 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.84.148:57917
Aug 06, 2026 12:45:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:45:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:45:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:45:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:45:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:45:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 00:45:26 dist-test-slave-n326 krb5kdc[13687](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977126, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.84.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 00:45:27 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977126, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.84.148@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:45:27.031599 30034 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.551s user 0.018s sys 0.215s
I20260806 00:45:27.031725 30034 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 00:45:27.031827 30034 mini_ranger_kms.cc:401] 127.29.84.148:57917/kms/v1/keys
Aug 06 00:45:27 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977126, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.84.148@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
Aug 06 00:45:29 dist-test-slave-n326 krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:45:29.529685 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:36545
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:36545
--ranger_config_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:57917/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:45:29.643561 14351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:45:29.643808 14351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:45:29.643877 14351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:45:29.647753 14351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:45:29.647825 14351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:45:29.647862 14351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:45:29.647939 14351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:45:29.647976 14351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:45:29.653100 14351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.84.148:57917/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:36545
--ranger_config_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:36545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:45:29.654035 14351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:45:29.654969 14351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:45:29.663636 14351 server_base.cc:1034] running on GCE node
W20260806 00:45:29.663663 14360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:45:29.663669 14364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:45:29.663779 14361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:45:29.664392 14351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:45:29.665395 14351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:45:29.666585 14351 hybrid_clock.cc:648] HybridClock initialized: now 1785977129666559 us; error 39 us; skew 500 ppm
Aug 06 00:45:29 dist-test-slave-n326 krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:45:29.672894 14351 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:45:29.673651 14351 webserver.cc:460] Webserver started at http://127.29.84.190:33531/ using document root <none> and password file <none>
I20260806 00:45:29.674088 14351 fs_manager.cc:359] Metadata directory not provided
I20260806 00:45:29.674136 14351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:45:29.674280 14351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:45:29 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:45:29.722095 14351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "a922e0e13ba44c6dbf77bea32dbd9e54"
format_stamp: "Formatted at 2026-08-06 00:45:29 on dist-test-slave-n326"
server_key: "ae0d808443cae64f40e4b9cf81a214d4"
server_key_iv: "53ccdc7f133392621d42bf2ec362b1fb"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:29.722648 14351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "a922e0e13ba44c6dbf77bea32dbd9e54"
format_stamp: "Formatted at 2026-08-06 00:45:29 on dist-test-slave-n326"
server_key: "ae0d808443cae64f40e4b9cf81a214d4"
server_key_iv: "53ccdc7f133392621d42bf2ec362b1fb"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:29.726583 14351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:45:29.739229 14376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:29.740481 14351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:45:29.740638 14351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "a922e0e13ba44c6dbf77bea32dbd9e54"
format_stamp: "Formatted at 2026-08-06 00:45:29 on dist-test-slave-n326"
server_key: "ae0d808443cae64f40e4b9cf81a214d4"
server_key_iv: "53ccdc7f133392621d42bf2ec362b1fb"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:29.740749 14351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:45:29.769827 14351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:45:29.772240 14351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:45:29.772445 14351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:45:29.779870 14351 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:36545
I20260806 00:45:29.779896 14450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:36545 every 8 connection(s)
I20260806 00:45:29.780786 14351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:45:29.783619 14452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:45:29.786157 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 14351
I20260806 00:45:29.786307 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:45:29.786598 30034 external_mini_cluster.cc:1428] Setting key 8427aaae69e0cc656ace93e5ab883efe
I20260806 00:45:29.789237 14452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54: Bootstrap starting.
I20260806 00:45:29.792032 14452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54: Neither blocks nor log segments found. Creating new log.
I20260806 00:45:29.792865 14452 log.cc:824] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54: Log is configured to *not* fsync() on all Append() calls
I20260806 00:45:29.794752 14452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54: No bootstrap required, opened a new log
Aug 06 00:45:29 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:45:29.797789 14452 raft_consensus.cc:348] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a922e0e13ba44c6dbf77bea32dbd9e54" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36545 } }
I20260806 00:45:29.797986 14452 raft_consensus.cc:374] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:45:29.798069 14452 raft_consensus.cc:729] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a922e0e13ba44c6dbf77bea32dbd9e54, State: Initialized, Role: FOLLOWER
I20260806 00:45:29.798578 14452 consensus_queue.cc:260] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a922e0e13ba44c6dbf77bea32dbd9e54" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36545 } }
I20260806 00:45:29.798744 14452 raft_consensus.cc:388] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:45:29.798856 14452 raft_consensus.cc:482] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:45:29.798982 14452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:45:29.799978 14452 raft_consensus.cc:504] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a922e0e13ba44c6dbf77bea32dbd9e54" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36545 } }
I20260806 00:45:29.800333 14452 leader_election.cc:302] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a922e0e13ba44c6dbf77bea32dbd9e54; no voters:
I20260806 00:45:29.800683 14452 leader_election.cc:288] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:45:29.800741 14460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:45:29.800921 14460 raft_consensus.cc:686] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [term 1 LEADER]: Becoming Leader. State: Replica: a922e0e13ba44c6dbf77bea32dbd9e54, State: Running, Role: LEADER
I20260806 00:45:29.801247 14460 consensus_queue.cc:237] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a922e0e13ba44c6dbf77bea32dbd9e54" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36545 } }
I20260806 00:45:29.801468 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:29.787887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40656 (local address 127.29.84.190:36545)
0806 00:45:29.788287 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:29.788302 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:29.788335 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:45:29.788907 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:29.788922 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:29.789216 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:29.789523 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:45:29.790406 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:29.791496 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:29.792687 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:29.793174 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:29.793367 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:29.796483 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:45:29.796492 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:45:29.796505 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:45:29.796529 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:45:29.798373 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:29.798850 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:29.798857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:29.798866 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:29.799018 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:29.799404 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:29.799406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:29.799407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:29.799759 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:45:29.799902 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:29.800268 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 00:45:29.800545 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":132,"threads_started":1}
I20260806 00:45:29.801877 14452 sys_catalog.cc:564] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:45:29.802850 14463 sys_catalog.cc:455] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a922e0e13ba44c6dbf77bea32dbd9e54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a922e0e13ba44c6dbf77bea32dbd9e54" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36545 } } }
I20260806 00:45:29.803269 14463 sys_catalog.cc:458] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [sys.catalog]: This master's current role is: LEADER
I20260806 00:45:29.803174 14467 sys_catalog.cc:455] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a922e0e13ba44c6dbf77bea32dbd9e54. Latest consensus state: current_term: 1 leader_uuid: "a922e0e13ba44c6dbf77bea32dbd9e54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a922e0e13ba44c6dbf77bea32dbd9e54" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 36545 } } }
I20260806 00:45:29.803633 14467 sys_catalog.cc:458] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54 [sys.catalog]: This master's current role is: LEADER
I20260806 00:45:29.804491 14479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:45:29.804860 14452 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 00:45:29.810123 14479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:45:29.817363 14479 catalog_manager.cc:1347] Generated new cluster ID: 64c84c3279ec409f82959852bd1fcf4b
I20260806 00:45:29.817423 14479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:45:29.834380 14479 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:45:29.835330 14479 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:45:29.841032 14479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54: Generated new TSK 0
I20260806 00:45:29.841674 14479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:45:30.516010 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:57917/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36545
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:45:30.649554 14520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:45:30.649832 14520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:45:30.649943 14520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:45:30.653966 14520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:45:30.654040 14520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:45:30.654126 14520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:45:30.659550 14520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.84.148:57917/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:45:30.660522 14520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:45:30.661616 14520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:45:30.671093 14527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:45:30.671150 14534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:45:30.671180 14529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:45:30.671356 14520 server_base.cc:1034] running on GCE node
I20260806 00:45:30.671913 14520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:45:30.672580 14520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:45:30.673758 14520 hybrid_clock.cc:648] HybridClock initialized: now 1785977130673740 us; error 35 us; skew 500 ppm
Aug 06 00:45:30 dist-test-slave-n326 krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977130, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:45:30.676582 14520 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:45:30.677364 14520 webserver.cc:460] Webserver started at http://127.29.84.129:36031/ using document root <none> and password file <none>
I20260806 00:45:30.677879 14520 fs_manager.cc:359] Metadata directory not provided
I20260806 00:45:30.677980 14520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:45:30.678226 14520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:45:30 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977130, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:45:30.696614 14520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "33701f2b3e00459bbe38b327203d4f19"
format_stamp: "Formatted at 2026-08-06 00:45:30 on dist-test-slave-n326"
server_key: "cce1f8778ccc538db0e3da02d8420c1b"
server_key_iv: "ca5a0fb2f5c8318cdb3c06ac22737b64"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:30.697307 14520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "33701f2b3e00459bbe38b327203d4f19"
format_stamp: "Formatted at 2026-08-06 00:45:30 on dist-test-slave-n326"
server_key: "cce1f8778ccc538db0e3da02d8420c1b"
server_key_iv: "ca5a0fb2f5c8318cdb3c06ac22737b64"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:30.702740 14520 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 00:45:30.718686 14546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:30.720289 14520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:45:30.720423 14520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "33701f2b3e00459bbe38b327203d4f19"
format_stamp: "Formatted at 2026-08-06 00:45:30 on dist-test-slave-n326"
server_key: "cce1f8778ccc538db0e3da02d8420c1b"
server_key_iv: "ca5a0fb2f5c8318cdb3c06ac22737b64"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:30.720559 14520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:45:30.740823 14520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:45:30.744319 14520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:45:30.744553 14520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:45:30.745281 14520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:45:30.746409 14520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:45:30.746516 14520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:30.746634 14520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:45:30.746719 14520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:30.757691 14520 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:36177
I20260806 00:45:30.757766 14721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:36177 every 8 connection(s)
I20260806 00:45:30.758836 14520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:45:30.763543 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 14520
I20260806 00:45:30.763686 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:45:30.764007 30034 external_mini_cluster.cc:1428] Setting key e6cbd25da6e679a79ac9f028f2682631
Aug 06 00:45:30 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977130, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:45:30.782681 14726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:30.761562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36545 (local address 127.29.84.129:44689)
0806 00:45:30.763282 (+ 1720us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:30.763335 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:45:30.764626 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:45:30.767793 (+ 3167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:45:30.767802 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:45:30.768130 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:45:30.768748 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:30.768768 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:30.769960 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:30.769963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:30.771405 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:30.771416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:30.772172 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:30.772176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:30.772360 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:30.775213 (+ 2853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:45:30.775234 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:45:30.777035 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:45:30.779279 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:45:30.779284 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:45:30.779298 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:45:30.779723 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:45:30.780134 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:45:30.780137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:45:30.780139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:45:30.780310 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:45:30.780819 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:45:30.780826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:45:30.781139 (+ 313us) client_negotiation.cc:769] Sending connection context
0806 00:45:30.781331 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 00:45:30.781700 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1475,"thread_start_us":143,"threads_started":1}
I20260806 00:45:30.783433 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:30.762704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:44689 (local address 127.29.84.190:36545)
0806 00:45:30.767283 (+ 4579us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:30.767295 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:30.767311 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:45:30.767353 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:30.767356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:30.767406 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:30.767497 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:45:30.768934 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:30.769793 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:30.771547 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:30.771909 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:30.772007 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:30.777232 (+ 5225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:45:30.777236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:45:30.777239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:45:30.777266 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:45:30.779154 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:30.779840 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:30.779844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:30.779845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:30.780019 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:30.780424 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:30.780427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:30.780429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:30.780668 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:45:30.780764 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:30.783151 (+ 2387us) server_negotiation.cc:299] Negotiation successful
0806 00:45:30.783278 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4496,"thread_start_us":107,"threads_started":1}
I20260806 00:45:30.786144 14722 heartbeater.cc:347] Connected to a master server at 127.29.84.190:36545
I20260806 00:45:30.786369 14722 heartbeater.cc:464] Registering TS with master...
I20260806 00:45:30.786918 14722 heartbeater.cc:511] Master 127.29.84.190:36545 requested a full tablet report, sending...
I20260806 00:45:30.788625 14396 ts_manager.cc:194] Registered new tserver with Master: 33701f2b3e00459bbe38b327203d4f19 (127.29.84.129:36177)
I20260806 00:45:30.790069 14396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:44689
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:45:30.845795 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:57917/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36545
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:45:30 dist-test-slave-n326 krb5kdc[13687](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977130, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 00:45:31.007472 14736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:45:31.007849 14736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:45:31.007962 14736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:45:31.012692 14736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:45:31.012771 14736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:45:31.012854 14736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:45:31.019961 14736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.84.148:57917/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:45:31.021284 14736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:45:31.022601 14736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:45:31.035313 14744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:45:31.035512 14743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:45:31.035318 14748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:45:31.036752 14736 server_base.cc:1034] running on GCE node
I20260806 00:45:31.037214 14736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:45:31.038113 14736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:45:31.039307 14736 hybrid_clock.cc:648] HybridClock initialized: now 1785977131039283 us; error 45 us; skew 500 ppm
Aug 06 00:45:31 dist-test-slave-n326 krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:45:31.043318 14736 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:45:31.044345 14736 webserver.cc:460] Webserver started at http://127.29.84.130:43649/ using document root <none> and password file <none>
I20260806 00:45:31.045028 14736 fs_manager.cc:359] Metadata directory not provided
I20260806 00:45:31.045117 14736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:45:31.045406 14736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:45:31 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:45:31.066161 14736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7"
format_stamp: "Formatted at 2026-08-06 00:45:31 on dist-test-slave-n326"
server_key: "be39473eeef9854d48a2e3f9dbb8af13"
server_key_iv: "4a03b513db9e265e87b17c4ee7555085"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:31.066877 14736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7"
format_stamp: "Formatted at 2026-08-06 00:45:31 on dist-test-slave-n326"
server_key: "be39473eeef9854d48a2e3f9dbb8af13"
server_key_iv: "4a03b513db9e265e87b17c4ee7555085"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:31.072184 14736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 00:45:31.100399 14759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:31.102121 14736 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.001s
I20260806 00:45:31.102249 14736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7"
format_stamp: "Formatted at 2026-08-06 00:45:31 on dist-test-slave-n326"
server_key: "be39473eeef9854d48a2e3f9dbb8af13"
server_key_iv: "4a03b513db9e265e87b17c4ee7555085"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:31.102385 14736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:45:31.127449 14736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:45:31.130967 14736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:45:31.131222 14736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:45:31.131928 14736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:45:31.132980 14736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:45:31.133080 14736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:31.133174 14736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:45:31.133234 14736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:31.147645 14736 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:46039
I20260806 00:45:31.148890 14736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:45:31.150343 14937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:46039 every 8 connection(s)
I20260806 00:45:31.150424 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 14736
I20260806 00:45:31.150589 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:45:31.150946 30034 external_mini_cluster.cc:1428] Setting key 94136d14c4d3af676288c9d3f1928539
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Aug 06 00:45:31 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:45:31.179494 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.157883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:49129 (local address 127.29.84.190:36545)
0806 00:45:31.158040 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:31.158046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:31.164530 (+ 6484us) server_negotiation.cc:407] Connection header received
0806 00:45:31.165926 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:31.165930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:31.165987 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:31.166042 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:45:31.167755 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.168407 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.169698 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.170119 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.170257 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.174681 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:45:31.174687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:45:31.174692 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:45:31.174730 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:45:31.176704 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:31.177411 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:31.177416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:31.177418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:31.177510 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:31.177989 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:31.177994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:31.177997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:31.178237 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:45:31.178331 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:31.179047 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 00:45:31.179256 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:45:31.181468 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.156815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36545 (local address 127.29.84.130:49129)
0806 00:45:31.164351 (+ 7536us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:31.164410 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 00:45:31.165651 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:45:31.166491 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:45:31.166503 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:45:31.166850 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:45:31.167560 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.167580 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.168543 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.168545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.169507 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.169516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.170435 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.170442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.170682 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.172584 (+ 1902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:45:31.172609 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:45:31.174444 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:45:31.176857 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:45:31.176866 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:45:31.176882 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:45:31.177266 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:45:31.177636 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:45:31.177642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:45:31.177644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:45:31.177858 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:45:31.178418 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:45:31.178426 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:45:31.178742 (+ 316us) client_negotiation.cc:769] Sending connection context
0806 00:45:31.180004 (+ 1262us) client_negotiation.cc:240] Negotiation successful
0806 00:45:31.180343 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7294,"thread_start_us":153,"threads_started":1}
I20260806 00:45:31.182802 14939 heartbeater.cc:347] Connected to a master server at 127.29.84.190:36545
I20260806 00:45:31.183171 14939 heartbeater.cc:464] Registering TS with master...
I20260806 00:45:31.183954 14939 heartbeater.cc:511] Master 127.29.84.190:36545 requested a full tablet report, sending...
I20260806 00:45:31.185175 14396 ts_manager.cc:194] Registered new tserver with Master: 2e1c2aa1342d4216a9c162d84c0c4bb7 (127.29.84.130:46039)
I20260806 00:45:31.185920 14396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:49129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:45:31.195243 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:57917/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36545
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:41067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:45:31.361091 14945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:45:31.361841 14945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:45:31.362008 14945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:45:31.368014 14945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:45:31.368141 14945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:45:31.368273 14945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:45:31.375619 14945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:41067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.84.148:57917/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:36545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:45:31.377061 14945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:45:31.378491 14945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:45:31.399402 14954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:45:31.399403 14953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:45:31.399958 14957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:45:31.400483 14945 server_base.cc:1034] running on GCE node
I20260806 00:45:31.401005 14945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:45:31.401851 14945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:45:31.403127 14945 hybrid_clock.cc:648] HybridClock initialized: now 1785977131403040 us; error 99 us; skew 500 ppm
Aug 06 00:45:31 dist-test-slave-n326 krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:45:31.406585 14945 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:45:31.407743 14945 webserver.cc:460] Webserver started at http://127.29.84.131:42695/ using document root <none> and password file <none>
I20260806 00:45:31.408550 14945 fs_manager.cc:359] Metadata directory not provided
I20260806 00:45:31.408648 14945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:45:31.408900 14945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:45:31 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
Aug 06 00:45:31 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977130, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:45:31.424863 14945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "427268e16e7244b7af13fdbf8e7ef8bc"
format_stamp: "Formatted at 2026-08-06 00:45:31 on dist-test-slave-n326"
server_key: "feaad7ab9523ca601e869c013bab3e93"
server_key_iv: "8674e9acf0b92f7ca132b84fc98ebecf"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:31.425631 14945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "427268e16e7244b7af13fdbf8e7ef8bc"
format_stamp: "Formatted at 2026-08-06 00:45:31 on dist-test-slave-n326"
server_key: "feaad7ab9523ca601e869c013bab3e93"
server_key_iv: "8674e9acf0b92f7ca132b84fc98ebecf"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:31.432464 14945 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 00:45:31.451486 14973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:31.453078 14945 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260806 00:45:31.453260 14945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "427268e16e7244b7af13fdbf8e7ef8bc"
format_stamp: "Formatted at 2026-08-06 00:45:31 on dist-test-slave-n326"
server_key: "feaad7ab9523ca601e869c013bab3e93"
server_key_iv: "8674e9acf0b92f7ca132b84fc98ebecf"
server_key_version: "kuduclusterkey@0"
I20260806 00:45:31.453459 14945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:45:31.472261 14945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:45:31.475728 14945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:45:31.475942 14945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:45:31.476680 14945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:45:31.477661 14945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:45:31.477764 14945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:31.477855 14945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:45:31.477916 14945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:45:31.488858 14945 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:38235
I20260806 00:45:31.490007 14945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:45:31.498131 15131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:38235 every 8 connection(s)
I20260806 00:45:31.499400 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 14945
I20260806 00:45:31.499514 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:45:31.499804 30034 external_mini_cluster.cc:1428] Setting key d480fd81bf09e04a34acb62b118114b9
Aug 06 00:45:31 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:45:31.519143 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.496684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:59159 (local address 127.29.84.190:36545)
0806 00:45:31.496812 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:31.496817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:31.500558 (+ 3741us) server_negotiation.cc:407] Connection header received
0806 00:45:31.501708 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:31.501711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:31.501759 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:31.501835 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:45:31.506295 (+ 4460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.507229 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.510868 (+ 3639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.511229 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.511311 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.514780 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:45:31.514784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:45:31.514787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:45:31.514814 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:45:31.516675 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:31.517307 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:31.517311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:31.517313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:31.517398 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:31.517805 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:31.517809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:31.517810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:31.518069 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:45:31.518146 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:31.518827 (+ 681us) server_negotiation.cc:299] Negotiation successful
0806 00:45:31.518948 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:45:31.520879 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.497167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:36545 (local address 127.29.84.131:59159)
0806 00:45:31.500410 (+ 3243us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:31.500463 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:45:31.501510 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:45:31.504462 (+ 2952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:45:31.504472 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:45:31.504830 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:45:31.505517 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.505536 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.508512 (+ 2976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.508516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.509917 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.509927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.511518 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.511522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.511714 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.512418 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:45:31.512438 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:45:31.514569 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:45:31.516821 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:45:31.516827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:45:31.516843 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:45:31.517186 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:45:31.517517 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:45:31.517520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:45:31.517522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:45:31.517693 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:45:31.518230 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:45:31.518236 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:45:31.518543 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 00:45:31.519526 (+ 983us) client_negotiation.cc:240] Negotiation successful
0806 00:45:31.519842 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3027,"thread_start_us":131,"threads_started":1}
I20260806 00:45:31.522410 15132 heartbeater.cc:347] Connected to a master server at 127.29.84.190:36545
I20260806 00:45:31.522665 15132 heartbeater.cc:464] Registering TS with master...
I20260806 00:45:31.523227 15132 heartbeater.cc:511] Master 127.29.84.190:36545 requested a full tablet report, sending...
I20260806 00:45:31.524335 14395 ts_manager.cc:194] Registered new tserver with Master: 427268e16e7244b7af13fdbf8e7ef8bc (127.29.84.131:38235)
I20260806 00:45:31.525085 14395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:59159
I20260806 00:45:31.525154 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:45:31.551843 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.535526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40664 (local address 127.29.84.190:36545)
0806 00:45:31.535630 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:31.535634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:31.538196 (+ 2562us) server_negotiation.cc:407] Connection header received
0806 00:45:31.538383 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:31.538386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:31.538433 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:31.538490 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:45:31.543735 (+ 5245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.544550 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.545968 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.546297 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.546373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.548250 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:45:31.548254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:45:31.548257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:45:31.548281 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:45:31.549956 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:31.550468 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:31.550472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:31.550474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:31.550555 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:45:31.550861 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:45:31.550865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:45:31.550866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:45:31.551096 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:45:31.551199 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:31.551546 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 00:45:31.551658 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:45:31.555683 14395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:45:31.559751 14395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:45:31.587914 15153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.578592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46039 (local address 127.0.0.1:55408)
0806 00:45:31.580583 (+ 1991us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:31.580631 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:45:31.580839 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:45:31.581609 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:45:31.581614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:45:31.581637 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:45:31.581758 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.582451 (+ 693us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.583995 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.583998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.586101 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.586112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.587381 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.587384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.587494 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.587529 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 00:45:31.587630 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 00:45:31.587755 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1823,"thread_start_us":106,"threads_started":1}
I20260806 00:45:31.588534 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.578784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55408 (local address 127.29.84.130:46039)
0806 00:45:31.580138 (+ 1354us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:31.580149 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:31.580735 (+ 586us) server_negotiation.cc:407] Connection header received
0806 00:45:31.581022 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:31.581028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:31.581232 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:31.581356 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:45:31.582575 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.583839 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.586257 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.587050 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.587150 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.587286 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:31.588292 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0806 00:45:31.588390 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1255,"thread_start_us":155,"threads_started":1}
I20260806 00:45:31.593927 14831 tablet_service.cc:1467] Processing CreateTablet for tablet 058f9a1212e04b0c9a2e844d39c16660 (DEFAULT_TABLE table=test-table [id=99af723dfb6641c7acb34e01ae86a06c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:45:31.595963 14831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058f9a1212e04b0c9a2e844d39c16660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:45:31.606865 15156 tablet_bootstrap.cc:492] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: Bootstrap starting.
I20260806 00:45:31.612902 15152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.579113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:36177 (local address 127.0.0.1:40906)
0806 00:45:31.589724 (+ 10611us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:31.589739 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:45:31.597348 (+ 7609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:45:31.606455 (+ 9107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:45:31.606458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:45:31.606478 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:45:31.608504 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.608512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.609988 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.609991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.611233 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.611244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.612483 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.612486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.612589 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.612612 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:45:31.612670 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 00:45:31.612738 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10550,"mutex_wait_us":1876,"thread_start_us":86,"threads_started":1}
I20260806 00:45:31.613438 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.579324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40906 (local address 127.29.84.129:36177)
0806 00:45:31.579933 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:31.579943 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:31.589832 (+ 9889us) server_negotiation.cc:407] Connection header received
0806 00:45:31.597500 (+ 7668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:31.597506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:31.597692 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:31.597831 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:45:31.608657 (+ 10826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.609837 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.611377 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.612186 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.612286 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.612420 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:31.613201 (+ 781us) server_negotiation.cc:299] Negotiation successful
0806 00:45:31.613304 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":134,"threads_started":1}
I20260806 00:45:31.615806 15150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.579342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:38235 (local address 127.0.0.1:55698)
0806 00:45:31.589974 (+ 10632us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:31.589986 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:45:31.590068 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:45:31.597469 (+ 7401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:45:31.597472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:45:31.597485 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:45:31.597572 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.597577 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.599052 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.599056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.613868 (+ 14812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.613880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.615440 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.615444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.615533 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.615557 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:45:31.615620 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 00:45:31.615684 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10582,"thread_start_us":128,"threads_started":1}
I20260806 00:45:31.616252 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.579793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.29.84.131:38235)
0806 00:45:31.580337 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:31.580347 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:31.590162 (+ 9815us) server_negotiation.cc:407] Connection header received
0806 00:45:31.590242 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:31.590247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:31.590441 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:31.590587 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:45:31.597697 (+ 7110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.598902 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.614075 (+ 15173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.615143 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.615250 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.615376 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:31.616005 (+ 629us) server_negotiation.cc:299] Negotiation successful
0806 00:45:31.616109 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":162,"threads_started":1}
I20260806 00:45:31.616758 14622 tablet_service.cc:1467] Processing CreateTablet for tablet 058f9a1212e04b0c9a2e844d39c16660 (DEFAULT_TABLE table=test-table [id=99af723dfb6641c7acb34e01ae86a06c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:45:31.617853 14622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058f9a1212e04b0c9a2e844d39c16660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:45:31.622022 15009 tablet_service.cc:1467] Processing CreateTablet for tablet 058f9a1212e04b0c9a2e844d39c16660 (DEFAULT_TABLE table=test-table [id=99af723dfb6641c7acb34e01ae86a06c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:45:31.622968 15009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058f9a1212e04b0c9a2e844d39c16660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:45:31.633046 15161 tablet_bootstrap.cc:492] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: Bootstrap starting.
I20260806 00:45:31.634388 15156 tablet_bootstrap.cc:654] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: Neither blocks nor log segments found. Creating new log.
I20260806 00:45:31.635394 15156 log.cc:824] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: Log is configured to *not* fsync() on all Append() calls
I20260806 00:45:31.636224 15157 tablet_bootstrap.cc:492] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19: Bootstrap starting.
I20260806 00:45:31.636631 15161 tablet_bootstrap.cc:654] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: Neither blocks nor log segments found. Creating new log.
I20260806 00:45:31.637530 15161 log.cc:824] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: Log is configured to *not* fsync() on all Append() calls
I20260806 00:45:31.638358 15157 tablet_bootstrap.cc:654] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19: Neither blocks nor log segments found. Creating new log.
I20260806 00:45:31.639210 15157 log.cc:824] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19: Log is configured to *not* fsync() on all Append() calls
I20260806 00:45:31.639638 15161 tablet_bootstrap.cc:492] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: No bootstrap required, opened a new log
I20260806 00:45:31.639892 15161 ts_tablet_manager.cc:1397] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260806 00:45:31.641337 15156 tablet_bootstrap.cc:492] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: No bootstrap required, opened a new log
I20260806 00:45:31.641572 15156 ts_tablet_manager.cc:1397] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.000s
I20260806 00:45:31.643991 15161 raft_consensus.cc:348] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } }
I20260806 00:45:31.644343 15161 raft_consensus.cc:374] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:45:31.644455 15161 raft_consensus.cc:729] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 427268e16e7244b7af13fdbf8e7ef8bc, State: Initialized, Role: FOLLOWER
I20260806 00:45:31.645103 15161 consensus_queue.cc:260] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } }
I20260806 00:45:31.645953 15156 raft_consensus.cc:348] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } }
I20260806 00:45:31.646193 15156 raft_consensus.cc:374] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:45:31.646240 15156 raft_consensus.cc:729] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1c2aa1342d4216a9c162d84c0c4bb7, State: Initialized, Role: FOLLOWER
I20260806 00:45:31.646659 15156 consensus_queue.cc:260] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } }
I20260806 00:45:31.647184 15161 ts_tablet_manager.cc:1428] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260806 00:45:31.647572 15132 heartbeater.cc:503] Master 127.29.84.190:36545 was elected leader, sending a full tablet report...
I20260806 00:45:31.647636 15156 ts_tablet_manager.cc:1428] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260806 00:45:31.647933 14939 heartbeater.cc:503] Master 127.29.84.190:36545 was elected leader, sending a full tablet report...
I20260806 00:45:31.649369 15157 tablet_bootstrap.cc:492] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19: No bootstrap required, opened a new log
I20260806 00:45:31.649578 15157 ts_tablet_manager.cc:1397] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.003s
W20260806 00:45:31.651638 14941 tablet.cc:2369] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:45:31.653965 15157 raft_consensus.cc:348] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } }
I20260806 00:45:31.654330 15157 raft_consensus.cc:374] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:45:31.654402 15157 raft_consensus.cc:729] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33701f2b3e00459bbe38b327203d4f19, State: Initialized, Role: FOLLOWER
I20260806 00:45:31.654855 15157 consensus_queue.cc:260] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } }
I20260806 00:45:31.655539 14722 heartbeater.cc:503] Master 127.29.84.190:36545 was elected leader, sending a full tablet report...
I20260806 00:45:31.655830 15157 ts_tablet_manager.cc:1428] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260806 00:45:31.685652 15166 raft_consensus.cc:482] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:45:31.685838 15166 raft_consensus.cc:504] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } }
I20260806 00:45:31.687022 15166 leader_election.cc:288] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e1c2aa1342d4216a9c162d84c0c4bb7 (127.29.84.130:46039), 33701f2b3e00459bbe38b327203d4f19 (127.29.84.129:36177)
I20260806 00:45:31.692339 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.687372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46039 (local address 127.29.84.131:54747)
0806 00:45:31.687667 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:31.687681 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:45:31.687757 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:45:31.688471 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:45:31.688474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:45:31.688496 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:45:31.688631 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.688638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.689630 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.689635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.690893 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.690904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.691901 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.691906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.692013 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.692056 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 00:45:31.692138 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 00:45:31.692209 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":44}
I20260806 00:45:31.693027 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.687804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:54747 (local address 127.29.84.130:46039)
0806 00:45:31.687931 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:31.687935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:31.687947 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:45:31.687991 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:31.687994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:31.688035 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:31.688102 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:45:31.688786 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.689488 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.691055 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.691735 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.691920 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.692007 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:31.692812 (+ 805us) server_negotiation.cc:299] Negotiation successful
0806 00:45:31.692912 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:45:31.693552 14860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058f9a1212e04b0c9a2e844d39c16660" candidate_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" is_pre_election: true
I20260806 00:45:31.693816 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.687947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:33877 (local address 127.29.84.129:36177)
0806 00:45:31.688282 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:31.688286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:31.688301 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:45:31.688345 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:31.688347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:31.688407 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:31.688482 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:45:31.688974 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.689734 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.692528 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.693154 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.693244 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.693286 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:31.693636 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 00:45:31.693709 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239}
I20260806 00:45:31.693820 14860 raft_consensus.cc:2454] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 427268e16e7244b7af13fdbf8e7ef8bc in term 0.
I20260806 00:45:31.694213 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.687401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:36177 (local address 127.29.84.131:33877)
0806 00:45:31.688162 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:31.688177 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:45:31.688272 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:45:31.688588 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:45:31.688591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:45:31.688611 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:45:31.688808 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.688815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.689910 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.689914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.692374 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:45:31.692386 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.693395 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.693398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:45:31.693494 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.693518 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:45:31.693998 (+ 480us) client_negotiation.cc:240] Negotiation successful
0806 00:45:31.694069 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":5,"thread_start_us":116,"threads_started":1}
I20260806 00:45:31.694787 14652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058f9a1212e04b0c9a2e844d39c16660" candidate_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33701f2b3e00459bbe38b327203d4f19" is_pre_election: true
I20260806 00:45:31.694782 14974 leader_election.cc:302] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e1c2aa1342d4216a9c162d84c0c4bb7, 427268e16e7244b7af13fdbf8e7ef8bc; no voters:
I20260806 00:45:31.695036 14652 raft_consensus.cc:2454] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 427268e16e7244b7af13fdbf8e7ef8bc in term 0.
I20260806 00:45:31.695122 15166 raft_consensus.cc:2781] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:45:31.695209 15166 raft_consensus.cc:482] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:45:31.695257 15166 raft_consensus.cc:3037] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:45:31.696841 15166 raft_consensus.cc:504] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } }
I20260806 00:45:31.697216 15166 leader_election.cc:288] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [CANDIDATE]: Term 1 election: Requested vote from peers 2e1c2aa1342d4216a9c162d84c0c4bb7 (127.29.84.130:46039), 33701f2b3e00459bbe38b327203d4f19 (127.29.84.129:36177)
I20260806 00:45:31.697970 14652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058f9a1212e04b0c9a2e844d39c16660" candidate_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33701f2b3e00459bbe38b327203d4f19"
I20260806 00:45:31.697997 14860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058f9a1212e04b0c9a2e844d39c16660" candidate_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7"
I20260806 00:45:31.698097 14652 raft_consensus.cc:3037] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:45:31.698108 14860 raft_consensus.cc:3037] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:45:31.699376 14860 raft_consensus.cc:2454] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 427268e16e7244b7af13fdbf8e7ef8bc in term 1.
I20260806 00:45:31.699388 14652 raft_consensus.cc:2454] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 427268e16e7244b7af13fdbf8e7ef8bc in term 1.
I20260806 00:45:31.699692 14974 leader_election.cc:302] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e1c2aa1342d4216a9c162d84c0c4bb7, 427268e16e7244b7af13fdbf8e7ef8bc; no voters:
I20260806 00:45:31.699918 15166 raft_consensus.cc:2781] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:45:31.700150 15166 raft_consensus.cc:686] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 1 LEADER]: Becoming Leader. State: Replica: 427268e16e7244b7af13fdbf8e7ef8bc, State: Running, Role: LEADER
I20260806 00:45:31.700482 15166 consensus_queue.cc:237] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } }
I20260806 00:45:31.704011 14395 catalog_manager.cc:5495] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 427268e16e7244b7af13fdbf8e7ef8bc (127.29.84.131). New cstate: current_term: 1 leader_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "427268e16e7244b7af13fdbf8e7ef8bc" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 38235 } health_report { overall_health: HEALTHY } } }
I20260806 00:45:31.745040 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.739597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.29.84.131:38235)
0806 00:45:31.739712 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 00:45:31.739717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:45:31.739920 (+ 203us) server_negotiation.cc:407] Connection header received
0806 00:45:31.740027 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:45:31.740030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:45:31.740094 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:45:31.740195 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:45:31.740590 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.741409 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.742741 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:45:31.743136 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:45:31.743238 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:45:31.743688 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:45:31.744263 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:45:31.744379 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:45:31.744835 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 00:45:31.744905 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 00:45:31.745448 15133 tablet.cc:2369] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:45:31.752918 14652 raft_consensus.cc:1261] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19 [term 1 FOLLOWER]: Refusing update from remote peer 427268e16e7244b7af13fdbf8e7ef8bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:45:31.753707 15166 consensus_queue.cc:1035] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "33701f2b3e00459bbe38b327203d4f19" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:45:31.756112 14860 raft_consensus.cc:1261] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [term 1 FOLLOWER]: Refusing update from remote peer 427268e16e7244b7af13fdbf8e7ef8bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:45:31.757963 15166 consensus_queue.cc:1035] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e1c2aa1342d4216a9c162d84c0c4bb7" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:45:31.764819 14723 tablet.cc:2369] T 058f9a1212e04b0c9a2e844d39c16660 P 33701f2b3e00459bbe38b327203d4f19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:45:31.775766 15174 mvcc.cc:205] Tried to move back new op lower bound from 7315362331651653632 to 7315362331447885824. Current Snapshot: MvccSnapshot[applied={T|T < 7315362331651653632}]
I20260806 00:45:31.779141 15177 mvcc.cc:205] Tried to move back new op lower bound from 7315362331651653632 to 7315362331447885824. Current Snapshot: MvccSnapshot[applied={T|T < 7315362331651653632}]
I20260806 00:45:31.789247 14395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:45:31.789414 14395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:45:31.792969 14395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a922e0e13ba44c6dbf77bea32dbd9e54: Sending DeleteTablet for 3 replicas of tablet 058f9a1212e04b0c9a2e844d39c16660
I20260806 00:45:31.793665 14622 tablet_service.cc:1514] Processing DeleteTablet for tablet 058f9a1212e04b0c9a2e844d39c16660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:45:31 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:40906
I20260806 00:45:31.794399 14831 tablet_service.cc:1514] Processing DeleteTablet for tablet 058f9a1212e04b0c9a2e844d39c16660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:45:31 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:55408
I20260806 00:45:31.795815 15009 tablet_service.cc:1514] Processing DeleteTablet for tablet 058f9a1212e04b0c9a2e844d39c16660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:45:31 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:55698
I20260806 00:45:31.796519 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 14520
Aug 06 00:45:31 dist-test-slave-n326 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977130, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:45:31.802531 15191 tablet_replica.cc:330] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: stopping tablet replica
I20260806 00:45:31.802851 15191 raft_consensus.cc:2229] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:45:31.803504 15191 raft_consensus.cc:2258] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:45:31.803841 15193 tablet_replica.cc:330] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: stopping tablet replica
I20260806 00:45:31.804194 15193 raft_consensus.cc:2229] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:45:31.804557 15193 raft_consensus.cc:2258] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:45:31.815815 15191 ts_tablet_manager.cc:1905] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:45:31.818513 15191 ts_tablet_manager.cc:1918] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:45:31.818605 15191 log.cc:1196] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/058f9a1212e04b0c9a2e844d39c16660
I20260806 00:45:31.818925 15191 ts_tablet_manager.cc:1939] T 058f9a1212e04b0c9a2e844d39c16660 P 2e1c2aa1342d4216a9c162d84c0c4bb7: Deleting consensus metadata
I20260806 00:45:31.820122 14377 catalog_manager.cc:4841] TS 2e1c2aa1342d4216a9c162d84c0c4bb7 (127.29.84.130:46039): tablet 058f9a1212e04b0c9a2e844d39c16660 (table test-table [id=99af723dfb6641c7acb34e01ae86a06c]) successfully deleted
W20260806 00:45:31.826222 14975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:36177 (error 108)
I20260806 00:45:31.827886 15193 ts_tablet_manager.cc:1905] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:45:31.830505 15193 ts_tablet_manager.cc:1918] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:45:31.830610 15193 log.cc:1196] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/058f9a1212e04b0c9a2e844d39c16660
I20260806 00:45:31.830974 15193 ts_tablet_manager.cc:1939] T 058f9a1212e04b0c9a2e844d39c16660 P 427268e16e7244b7af13fdbf8e7ef8bc: Deleting consensus metadata
I20260806 00:45:31.832098 14377 catalog_manager.cc:4841] TS 427268e16e7244b7af13fdbf8e7ef8bc (127.29.84.131:38235): tablet 058f9a1212e04b0c9a2e844d39c16660 (table test-table [id=99af723dfb6641c7acb34e01ae86a06c]) successfully deleted
W20260806 00:45:31.832631 14379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:36177 (error 108)
W20260806 00:45:31.832701 15170 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:45:31.831565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:36177 (local address 127.29.84.131:33425)
0806 00:45:31.831680 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:31.831695 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:45:31.831778 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:45:31.832604 (+ 826us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:36177: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 00:45:31.833405 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 14736
I20260806 00:45:31.841964 15152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:45:31.841630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:36177 (local address 127.0.0.1:40914)
0806 00:45:31.841800 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 00:45:31.841866 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:36177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260806 00:45:31.842214 14379 catalog_manager.cc:4584] TS 33701f2b3e00459bbe38b327203d4f19 (127.29.84.129:36177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 058f9a1212e04b0c9a2e844d39c16660: Network error: Client connection negotiation failed: client connection to 127.29.84.129:36177: connect: Connection refused (error 111)
I20260806 00:45:31.844791 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 14945
I20260806 00:45:31.857712 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 14351
I20260806 00:45:31.871532 30034 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 00:45:32.226248 30034 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 00:45:32.226408 30034 mini_ranger.cc:66] Stopping Ranger...
I20260806 00:45:32.570572 30034 mini_ranger.cc:68] Stopped Ranger
2026-08-06 00:45:32.570 UTC [13722] LOG: received smart shutdown request
2026-08-06 00:45:32.574 UTC [13722] LOG: background worker "logical replication launcher" (PID 13733) exited with exit code 1
2026-08-06 00:45:32.575 UTC [13728] LOG: shutting down
2026-08-06 00:45:32.732 UTC [13722] LOG: database system is shut down
2026-08-06T00:45:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46650 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:45:32 dist-test-slave-n326 krb5kdc[15207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:45:32 dist-test-slave-n326 krb5kdc[15207](info): set up 2 sockets
Aug 06 00:45:32 dist-test-slave-n326 krb5kdc[15207](info): commencing operation
krb5kdc: starting...
W20260806 00:45:34.057858 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:45:34 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:45:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:45:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.84.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.84.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/rangeradmin_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/rangeradmin_127.29.84.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.84.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.84.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/rangerlookup_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/rangerlookup_127.29.84.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.84.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.84.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/HTTP_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/HTTP_127.29.84.148@KRBTEST.COM.keytab.
I20260806 00:45:34.225572 30034 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskdVRgL9/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/postgres -l logfile start
2026-08-06 00:45:36.558 UTC [15247] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 00:45:36.558 UTC [15247] LOG: listening on IPv4 address "127.29.84.148", port 53495
2026-08-06 00:45:36.562 UTC [15247] LOG: listening on Unix socket "/tmp/.s.PGSQL.53495"
2026-08-06 00:45:36.584 UTC [15252] LOG: database system was shut down at 2026-08-06 00:45:35 UTC
2026-08-06 00:45:36.588 UTC [15247] LOG: database system is ready to accept connections
I20260806 00:45:37.688817 30034 mini_postgres.cc:96] Postgres bound to 53495
2026-08-06 00:45:37.696 UTC [15262] FATAL: database "slave" does not exist
127.29.84.148:53495 - accepting connections
I20260806 00:45:37.696844 30034 mini_ranger.cc:161] Starting Ranger...
I20260806 00:45:37.712234 30034 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 00:45:38.162873 30034 mini_ranger.cc:87] Created ranger Postgres database
I20260806 00:45:38.162948 30034 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-admin
2026-08-06 00:45:38,447 [I] DB FLAVOR :POSTGRES
2026-08-06 00:45:38,448 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:45:38,448 [I] Checking connection..
2026-08-06 00:45:38,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 00:45:38,694 [I] Checking connection passed.
2026-08-06 00:45:38,695 [I] --------- Verifying version history table ---------
2026-08-06 00:45:38,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:45:38,957 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:45:38,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:45:39,215 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:45:39,216 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 00:45:39,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 00:45:39.452 UTC [15351] WARNING: there is no transaction in progress
2026-08-06 00:45:39,458 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 00:45:39,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:45:39,716 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 00:45:39,717 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:45:39,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 00:45:39,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 00:45:40,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-n326.c.gcp-upstream.internal','N') ;"
2026-08-06 00:45:40,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:45:40,691 [I] Table x_portal_user does not exist in database ranger
2026-08-06 00:45:40,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:45:40,938 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 00:45:40,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:45:41,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:45:41,414 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 00:45:41,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 00:45:42.328 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.343 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.359 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.373 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.388 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.403 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.461 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.468 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.481 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.491 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.500 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.513 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.522 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.528 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.534 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:42.540 UTC [15519] WARNING: there is no transaction in progress
2026-08-06 00:45:43,165 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 00:45:43,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 00:45:43,398 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 00:45:43,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:45:43,646 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 00:45:43,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:45:43,900 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 00:45:43,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:45:44,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:45:44,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-n326.c.gcp-upstream.internal';"
2026-08-06 00:45:44,586 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 00:45:44,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 00:45:44,828 [I] DB_PATCHES have already been applied
I20260806 00:45:44.834120 30034 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-admin:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskdVRgL9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 00:45:44.834177 30034 mini_ranger.cc:193] Using host: 127.29.84.148
I20260806 00:45:44.838150 30034 mini_ranger.cc:227] Ranger admin URL: http://127.29.84.148:55799
Aug 06, 2026 12:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 00:45:46.156985 30034 mini_ranger.cc:160] Time spent starting Ranger: real 8.460s user 0.000s sys 0.011s
Aug 06, 2026 12:46:04 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:46:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 12:46:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:46:04 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:46:05 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 00:46:05.957394 30034 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.84.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.84.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/rangerkms_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.84.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/rangerkms_127.29.84.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.84.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.84.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.84.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/HTTP_127.29.84.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.84.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/HTTP_127.29.84.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 00:46:06 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977166, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 00:46:06.038450 30034 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 00:46:06.048143 30034 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 00:46:06.897212 30034 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 00:46:06.897348 30034 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-kms
I20260806 00:46:06.897373 30034 mini_ranger_kms.cc:227] Using postgres at 127.29.84.148:53495
2026-08-06 00:46:07,071 [I] DB FLAVOR :POSTGRES
2026-08-06 00:46:07,072 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:46:07,072 [I] Checking connection
2026-08-06 00:46:07,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 00:46:07,317 [I] connection success
2026-08-06 00:46:07,318 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 00:46:07,318 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 00:46:07,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 00:46:07,543 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 00:46:07,544 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:46:07,544 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 00:46:07,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.84.148:53495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 00:46:07,788 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 00:46:08.227779 30034 mini_ranger_kms.cc:310] Created kms service
I20260806 00:46:08.437638 30034 mini_ranger_kms.cc:326] Created kudu user
I20260806 00:46:08.480238 30034 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 00:46:08 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977166, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:08.876104 30034 mini_ranger_kms.cc:384] Added ranger policy
I20260806 00:46:08.876281 30034 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-kms:/tmp/dist-test-taskdVRgL9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskdVRgL9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 00:46:08.876312 30034 mini_ranger_kms.cc:240] Using host: 127.29.84.148
I20260806 00:46:08.881539 30034 mini_ranger_kms.cc:276] Ranger KMS PID: 15794
I20260806 00:46:08.881621 30034 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.84.148:50517
Aug 06, 2026 12:46:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:46:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:46:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:46:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:46:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:46:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 00:46:10 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977170, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.84.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:11.561700 30034 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.523s user 0.005s sys 0.154s
I20260806 00:46:11.561851 30034 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 00:46:11.561931 30034 mini_ranger_kms.cc:401] 127.29.84.148:50517/kms/v1/keys
Aug 06 00:46:11 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977170, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.84.148@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
Aug 06 00:46:11 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977170, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.84.148@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
Aug 06 00:46:14 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:46:14.168226 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35323
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:35323
--ranger_config_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:50517/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:14.284376 15874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:14.284737 15874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:14.284801 15874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:14.291242 15874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:14.291332 15874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:14.291373 15874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:14.291425 15874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:14.291451 15874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:14.296722 15874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.84.148:50517/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:35323
--ranger_config_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:14.297816 15874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:14.298868 15874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:14.307399 15883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:14.307494 15885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:14.307476 15888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:14.307701 15874 server_base.cc:1034] running on GCE node
I20260806 00:46:14.308284 15874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:14.309280 15874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:14.310437 15874 hybrid_clock.cc:648] HybridClock initialized: now 1785977174310419 us; error 28 us; skew 500 ppm
Aug 06 00:46:14 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977174, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:14.313148 15874 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:46:14.314024 15874 webserver.cc:460] Webserver started at http://127.29.84.190:38525/ using document root <none> and password file <none>
I20260806 00:46:14.314509 15874 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:14.314563 15874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:14.314720 15874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:46:14 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977174, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:14.374554 15874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "1ee08a50cc24418e804499b4cea3f3ef"
format_stamp: "Formatted at 2026-08-06 00:46:14 on dist-test-slave-n326"
server_key: "3564ee0c17fe587441f13061bf999c35"
server_key_iv: "320861af2a120ee6e5900f53e294b2f9"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:14.375293 15874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "1ee08a50cc24418e804499b4cea3f3ef"
format_stamp: "Formatted at 2026-08-06 00:46:14 on dist-test-slave-n326"
server_key: "3564ee0c17fe587441f13061bf999c35"
server_key_iv: "320861af2a120ee6e5900f53e294b2f9"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:14.380127 15874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 00:46:14.396559 15909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:14.398144 15874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:46:14.398278 15874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "1ee08a50cc24418e804499b4cea3f3ef"
format_stamp: "Formatted at 2026-08-06 00:46:14 on dist-test-slave-n326"
server_key: "3564ee0c17fe587441f13061bf999c35"
server_key_iv: "320861af2a120ee6e5900f53e294b2f9"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:14.398404 15874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:14.421974 15874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:14.424539 15874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:14.424743 15874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:14.431567 15874 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:35323
I20260806 00:46:14.431603 15991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:35323 every 8 connection(s)
I20260806 00:46:14.432518 15874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:46:14.435253 15992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:14.435302 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 15874
I20260806 00:46:14.435472 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:46:14.435778 30034 external_mini_cluster.cc:1428] Setting key 1f4ec4263dd4725e6bdb1a4b95b3b61f
I20260806 00:46:14.441893 15992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Bootstrap starting.
I20260806 00:46:14.444583 15992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Neither blocks nor log segments found. Creating new log.
Aug 06 00:46:14 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:14.445341 15992 log.cc:824] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:14.447113 15992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: No bootstrap required, opened a new log
I20260806 00:46:14.449965 15992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } }
I20260806 00:46:14.450170 15992 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:14.450227 15992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee08a50cc24418e804499b4cea3f3ef, State: Initialized, Role: FOLLOWER
I20260806 00:46:14.450356 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:14.437057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.29.84.190:35323)
0806 00:46:14.437434 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:14.437450 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:14.437483 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:46:14.438337 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:14.438355 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:14.438685 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:14.438956 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:14.439924 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:14.440876 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:14.442048 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:14.442499 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:14.442616 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:14.445364 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:14.445368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:14.445381 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:14.445427 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:14.447668 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:14.448097 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:14.448103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:14.448108 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:14.448218 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:14.448509 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:14.448514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:14.448515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:14.448869 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:14.449073 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:14.449352 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 00:46:14.449624 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":122,"threads_started":1}
I20260806 00:46:14.450681 15992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } }
I20260806 00:46:14.450834 15992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:46:14.450917 15992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:46:14.451014 15992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:14.451839 15992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } }
I20260806 00:46:14.452179 15992 leader_election.cc:302] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ee08a50cc24418e804499b4cea3f3ef; no voters:
I20260806 00:46:14.452500 15992 leader_election.cc:288] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:46:14.452621 15997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:46:14.452826 15997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 1 LEADER]: Becoming Leader. State: Replica: 1ee08a50cc24418e804499b4cea3f3ef, State: Running, Role: LEADER
I20260806 00:46:14.453162 15997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } }
I20260806 00:46:14.453655 15992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:14.454814 15999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee08a50cc24418e804499b4cea3f3ef. Latest consensus state: current_term: 1 leader_uuid: "1ee08a50cc24418e804499b4cea3f3ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } } }
I20260806 00:46:14.454941 15999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:14.455237 15998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ee08a50cc24418e804499b4cea3f3ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } } }
I20260806 00:46:14.455324 15998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:14.455327 16009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:46:14.458570 16009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:46:14.458961 15992 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 00:46:14.468446 16009 catalog_manager.cc:1347] Generated new cluster ID: b33aae67087041c98388b0e22e53dc42
I20260806 00:46:14.468523 16009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:46:14.503527 16009 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:46:14.504837 16009 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:46:14.517417 16009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Generated new TSK 0
I20260806 00:46:14.518165 16009 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:46:15.155735 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:50517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35323
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:15.271795 16059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:15.272055 16059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:15.272166 16059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:15.276366 16059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:15.276434 16059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:15.276539 16059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:46:15.282086 16059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.84.148:50517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:15.283106 16059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:15.284114 16059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:15.293188 16067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:15.293210 16066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:15.293493 16069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:15.294065 16059 server_base.cc:1034] running on GCE node
I20260806 00:46:15.294435 16059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:15.295002 16059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:15.296226 16059 hybrid_clock.cc:648] HybridClock initialized: now 1785977175296195 us; error 42 us; skew 500 ppm
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:15.299521 16059 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:46:15.300518 16059 webserver.cc:460] Webserver started at http://127.29.84.129:36527/ using document root <none> and password file <none>
I20260806 00:46:15.301180 16059 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:15.301278 16059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:15.301550 16059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:15.319000 16059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "e52e6eac8aae4758979bb30fb2a30ec1"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "dd062f518b64c4404ba3ca113a40952f"
server_key_iv: "7e4a4283eab9ab3aaa87ff6afcad715d"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:15.319847 16059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "e52e6eac8aae4758979bb30fb2a30ec1"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "dd062f518b64c4404ba3ca113a40952f"
server_key_iv: "7e4a4283eab9ab3aaa87ff6afcad715d"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:15.325126 16059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 00:46:15.339481 16084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:15.341320 16059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 00:46:15.341473 16059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "e52e6eac8aae4758979bb30fb2a30ec1"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "dd062f518b64c4404ba3ca113a40952f"
server_key_iv: "7e4a4283eab9ab3aaa87ff6afcad715d"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:15.341678 16059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:15.401427 16059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:15.405021 16059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:15.405424 16059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:15.406190 16059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:15.407253 16059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:15.407321 16059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:15.407415 16059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:15.407452 16059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:15.419270 16059 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:44291
I20260806 00:46:15.420397 16059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:46:15.423214 16268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:44291 every 8 connection(s)
I20260806 00:46:15.423622 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 16059
I20260806 00:46:15.423749 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:46:15.424031 30034 external_mini_cluster.cc:1428] Setting key f72c057ba14eee6a6189e03b106abf05
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
I20260806 00:46:15.446797 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:15.425215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35323 (local address 127.29.84.129:45531)
0806 00:46:15.426348 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:15.426389 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:15.427168 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:15.431735 (+ 4567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:15.431745 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:15.432087 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:15.432798 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:15.432817 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:15.434108 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:15.434112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:15.435539 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:15.435551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:15.436206 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:15.436210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:15.436397 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:15.437585 (+ 1188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:15.437604 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:15.439956 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:15.444061 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:15.444067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:15.444080 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:15.444390 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:15.444673 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:15.444676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:15.444678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:15.444819 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:15.445211 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:15.445217 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:15.445470 (+ 253us) client_negotiation.cc:769] Sending connection context
0806 00:46:15.445625 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 00:46:15.445903 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":123,"threads_started":1}
I20260806 00:46:15.448140 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:15.426148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:45531 (local address 127.29.84.190:35323)
0806 00:46:15.431289 (+ 5141us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:15.431297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:15.431313 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:46:15.431354 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:15.431357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:15.431403 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:15.432949 (+ 1546us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:15.433057 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:15.433952 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:15.435690 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:15.436040 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:15.436716 (+ 676us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:15.440159 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:15.440163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:15.440166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:15.440190 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:15.442111 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:15.444503 (+ 2392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:15.444506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:15.444508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:15.444584 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:15.444914 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:15.444917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:15.444918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:15.445110 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:15.447707 (+ 2597us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:15.447794 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 00:46:15.447931 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5056,"thread_start_us":102,"threads_started":1}
I20260806 00:46:15.449494 16269 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35323
I20260806 00:46:15.449750 16269 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:15.450333 16269 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
I20260806 00:46:15.452239 15935 ts_manager.cc:194] Registered new tserver with Master: e52e6eac8aae4758979bb30fb2a30ec1 (127.29.84.129:44291)
I20260806 00:46:15.453850 15935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:45531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:46:15.462311 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:50517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35323
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 00:46:15.586570 16281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:15.586843 16281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:15.586911 16281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:15.591347 16281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:15.591449 16281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:15.591547 16281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:46:15.597249 16281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.84.148:50517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:15.598417 16281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:15.599471 16281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:15.609133 16295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:15.609180 16291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:15.609483 16281 server_base.cc:1034] running on GCE node
W20260806 00:46:15.609702 16290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:15.610137 16281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:15.610711 16281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:15.611904 16281 hybrid_clock.cc:648] HybridClock initialized: now 1785977175611866 us; error 49 us; skew 500 ppm
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:15.614607 16281 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:46:15.615552 16281 webserver.cc:460] Webserver started at http://127.29.84.130:37183/ using document root <none> and password file <none>
I20260806 00:46:15.616068 16281 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:15.616153 16281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:15.616354 16281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:15.633136 16281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "f0c578a02c6943c0a08538e50ce47fb2"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "85b51e1e7b3a33d7c1c6f55ea3c2e424"
server_key_iv: "ee956238db4b936b066f906ba664da6e"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:15.633898 16281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "f0c578a02c6943c0a08538e50ce47fb2"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "85b51e1e7b3a33d7c1c6f55ea3c2e424"
server_key_iv: "ee956238db4b936b066f906ba664da6e"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:15.641345 16281 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260806 00:46:15.659462 16304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:15.661103 16281 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260806 00:46:15.661298 16281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "f0c578a02c6943c0a08538e50ce47fb2"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "85b51e1e7b3a33d7c1c6f55ea3c2e424"
server_key_iv: "ee956238db4b936b066f906ba664da6e"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:15.661485 16281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:15.694828 16281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:15.697705 16281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:15.697889 16281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:15.698529 16281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:15.699429 16281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:15.699503 16281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:15.699571 16281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:15.699620 16281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:15.711344 16281 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:45807
I20260806 00:46:15.711584 16463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:45807 every 8 connection(s)
I20260806 00:46:15.712447 16281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:46:15.714795 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 16281
I20260806 00:46:15.714933 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:46:15.715298 30034 external_mini_cluster.cc:1428] Setting key af9f3434511019fdebecdf7489e8ce0e
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:15.730484 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:15.714813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35323 (local address 127.29.84.130:35983)
0806 00:46:15.716439 (+ 1626us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:15.716492 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:15.717516 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:15.718245 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:15.718254 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:15.718620 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:15.719320 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:15.719342 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:15.720477 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:15.720481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:15.721871 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:15.721883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:15.722578 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:15.722582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:15.722784 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:15.723453 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:15.723473 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:15.725270 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:15.727494 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:15.727499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:15.727512 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:15.727831 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:15.728147 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:15.728150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:15.728152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:15.728308 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:15.728783 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:15.728789 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:15.729079 (+ 290us) client_negotiation.cc:769] Sending connection context
0806 00:46:15.729259 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 00:46:15.729548 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1414,"thread_start_us":159,"threads_started":1}
I20260806 00:46:15.731187 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:15.714355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:35983 (local address 127.29.84.190:35323)
0806 00:46:15.714481 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:15.714485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:15.716595 (+ 2110us) server_negotiation.cc:407] Connection header received
0806 00:46:15.717755 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:15.717759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:15.717808 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:15.717857 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:15.719497 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:15.720310 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:15.722025 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:15.722396 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:15.722526 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:15.725523 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:15.725527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:15.725530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:15.725558 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:15.727361 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:15.727961 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:15.727965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:15.727967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:15.728051 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:15.728421 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:15.728424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:15.728426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:15.728646 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:15.728730 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:15.730880 (+ 2150us) server_negotiation.cc:299] Negotiation successful
0806 00:46:15.731017 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:46:15.732955 16465 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35323
I20260806 00:46:15.733174 16465 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:15.733686 16465 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
I20260806 00:46:15.734750 15935 ts_manager.cc:194] Registered new tserver with Master: f0c578a02c6943c0a08538e50ce47fb2 (127.29.84.130:45807)
I20260806 00:46:15.735610 15935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:35983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:46:15.760370 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:50517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35323
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:15.904277 16471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:15.904587 16471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:15.904645 16471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:15.910456 16471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:15.910526 16471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:15.910604 16471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:46:15.917761 16471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.84.148:50517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:35323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:15.918941 16471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:15.920243 16471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:15.931685 16477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:15.931704 16478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:15.931995 16482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:15.932652 16471 server_base.cc:1034] running on GCE node
I20260806 00:46:15.933074 16471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:15.933776 16471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:15.934950 16471 hybrid_clock.cc:648] HybridClock initialized: now 1785977175934889 us; error 76 us; skew 500 ppm
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:15.938387 16471 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:46:15.939406 16471 webserver.cc:460] Webserver started at http://127.29.84.131:33895/ using document root <none> and password file <none>
I20260806 00:46:15.940058 16471 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:15.940135 16471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:15.940336 16471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:15.957665 16471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "9067d6bf11f94fa18a1de8f372e0d610"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "0179e4d89fa0710240ccb92155484404"
server_key_iv: "43ec74dbe187f13890fdf7efd366a8cd"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:15.958350 16471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "9067d6bf11f94fa18a1de8f372e0d610"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "0179e4d89fa0710240ccb92155484404"
server_key_iv: "43ec74dbe187f13890fdf7efd366a8cd"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:15.966528 16471 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.000s
Aug 06 00:46:15 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:15.987733 16494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:15.989370 16471 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260806 00:46:15.989545 16471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "9067d6bf11f94fa18a1de8f372e0d610"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "0179e4d89fa0710240ccb92155484404"
server_key_iv: "43ec74dbe187f13890fdf7efd366a8cd"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:15.989760 16471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:16.006170 16471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:16.009948 16471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:16.010159 16471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:16.010805 16471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:16.011866 16471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:16.011999 16471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:16.012112 16471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:16.012173 16471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:16.023540 16471 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:45403
I20260806 00:46:16.024605 16471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:46:16.026260 16666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:45403 every 8 connection(s)
I20260806 00:46:16.030364 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 16471
I20260806 00:46:16.030539 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:46:16.030859 30034 external_mini_cluster.cc:1428] Setting key 2b53cef2b58a5b286ae6930b7f626e2e
Aug 06 00:46:16 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:16.045215 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:16.027834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:34781 (local address 127.29.84.190:35323)
0806 00:46:16.027945 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:16.027949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:16.028329 (+ 380us) server_negotiation.cc:407] Connection header received
0806 00:46:16.029659 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:16.029663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:16.029716 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:16.029768 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:16.033051 (+ 3283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.033840 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.035790 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.036169 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.036241 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:16.039871 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:16.039875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:16.039877 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:16.039917 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:16.042889 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:16.043513 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:16.043517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:16.043519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:16.043609 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:16.043991 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:16.043995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:16.043996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:16.044198 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:16.044276 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:16.044911 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 00:46:16.045056 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:46:16.046060 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:16.026839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35323 (local address 127.29.84.131:34781)
0806 00:46:16.028202 (+ 1363us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:16.028250 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:16.029403 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:16.030211 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:16.030220 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:16.030605 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:16.032879 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:16.032909 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.033998 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.034001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:16.035329 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:16.035339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.037005 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.037012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:16.037202 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:16.037835 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:16.037855 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:16.039672 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:16.043003 (+ 3331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:16.043007 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:16.043018 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:16.043386 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:16.043712 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:16.043715 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:16.043717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:16.043873 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:16.044338 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:16.044344 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:16.044621 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 00:46:16.044808 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 00:46:16.045114 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1174,"thread_start_us":148,"threads_started":1}
I20260806 00:46:16.047389 16667 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35323
I20260806 00:46:16.047626 16667 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:16.048216 16667 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
I20260806 00:46:16.049331 15935 ts_manager.cc:194] Registered new tserver with Master: 9067d6bf11f94fa18a1de8f372e0d610 (127.29.84.131:45403)
I20260806 00:46:16.050063 15935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:34781
I20260806 00:46:16.056245 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:46:16.057111 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 16059
I20260806 00:46:16.079548 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 16281
I20260806 00:46:16.087878 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 16471
I20260806 00:46:16.105645 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 15874
I20260806 00:46:16.117305 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35323
--webserver_interface=127.29.84.190
--webserver_port=38525
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:35323
--ranger_config_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:50517/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.84.148:50517/kms with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:16.228061 16672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:16.228293 16672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:16.228364 16672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:16.232190 16672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:16.232237 16672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:16.232255 16672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:16.232299 16672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:16.232328 16672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:16.237318 16672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.84.148:50517/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:35323
--ranger_config_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:35323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=38525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:16.238204 16672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:16.239156 16672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:16.247274 16680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:16.247275 16682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:16.247301 16684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:16.247359 16672 server_base.cc:1034] running on GCE node
I20260806 00:46:16.248071 16672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:16.249059 16672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:16.250243 16672 hybrid_clock.cc:648] HybridClock initialized: now 1785977176250232 us; error 29 us; skew 500 ppm
Aug 06 00:46:16 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:16.252964 16672 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:46:16.253711 16672 webserver.cc:460] Webserver started at http://127.29.84.190:38525/ using document root <none> and password file <none>
I20260806 00:46:16.254232 16672 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:16.254300 16672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:46:16 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:16.306185 16672 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:46:16.308614 16695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:16.309918 16672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:46:16.310068 16672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "1ee08a50cc24418e804499b4cea3f3ef"
format_stamp: "Formatted at 2026-08-06 00:46:14 on dist-test-slave-n326"
server_key: "3564ee0c17fe587441f13061bf999c35"
server_key_iv: "320861af2a120ee6e5900f53e294b2f9"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:16.310591 16672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:16.331866 16672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:16.334563 16672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:16.334746 16672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:16.341724 16672 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:35323
I20260806 00:46:16.341737 16765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:35323 every 8 connection(s)
I20260806 00:46:16.342695 16672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:46:16.344458 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 16672
I20260806 00:46:16.344964 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:50517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:44291
--local_ip_for_outbound_sockets=127.29.84.129
--tserver_master_addrs=127.29.84.190:35323
--webserver_port=36527
--webserver_interface=127.29.84.129
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.84.148:50517/kms with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:46:16.347427 16767 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:46:16.348980 16767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Bootstrap starting.
I20260806 00:46:16.364658 16767 log.cc:824] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:16.370666 16767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:46:16.371006 16767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Bootstrap complete.
I20260806 00:46:16.374802 16767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } }
I20260806 00:46:16.375201 16767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee08a50cc24418e804499b4cea3f3ef, State: Initialized, Role: FOLLOWER
I20260806 00:46:16.375646 16767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } }
I20260806 00:46:16.375747 16767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:46:16.375833 16767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:46:16.375938 16767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:46:16.377568 16767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } }
I20260806 00:46:16.377869 16767 leader_election.cc:302] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ee08a50cc24418e804499b4cea3f3ef; no voters:
I20260806 00:46:16.378175 16767 leader_election.cc:288] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:46:16.378247 16775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:46:16.378432 16775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [term 2 LEADER]: Becoming Leader. State: Replica: 1ee08a50cc24418e804499b4cea3f3ef, State: Running, Role: LEADER
I20260806 00:46:16.378731 16775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } }
I20260806 00:46:16.379354 16767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:16.379626 16777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1ee08a50cc24418e804499b4cea3f3ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } } }
I20260806 00:46:16.380326 16777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:16.380223 16776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee08a50cc24418e804499b4cea3f3ef. Latest consensus state: current_term: 2 leader_uuid: "1ee08a50cc24418e804499b4cea3f3ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee08a50cc24418e804499b4cea3f3ef" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 35323 } } }
I20260806 00:46:16.380651 16776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:16.380904 16788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:46:16.383973 16788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:46:16.384509 16767 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 00:46:16.384603 16788 catalog_manager.cc:1259] Loaded cluster ID: b33aae67087041c98388b0e22e53dc42
I20260806 00:46:16.384651 16788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:46:16.386874 16788 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:46:16.387589 16788 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Loaded TSK: 0
I20260806 00:46:16.388753 16788 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:46:16.483842 16769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:16.484184 16769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:16.484258 16769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:16.490568 16769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:16.490640 16769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:16.490725 16769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:46:16.498271 16769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.84.148:50517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:44291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=36527
--tserver_master_addrs=127.29.84.190:35323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:16.499444 16769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:16.500617 16769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:16.512717 16823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:16.512725 16826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:16.513154 16824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:16.513690 16769 server_base.cc:1034] running on GCE node
I20260806 00:46:16.514221 16769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:16.515028 16769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:16.516259 16769 hybrid_clock.cc:648] HybridClock initialized: now 1785977176516240 us; error 32 us; skew 500 ppm
Aug 06 00:46:16 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:16.521441 16769 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:46:16.522925 16769 webserver.cc:460] Webserver started at http://127.29.84.129:36527/ using document root <none> and password file <none>
I20260806 00:46:16.523787 16769 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:16.523898 16769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:46:16 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:16.552479 16769 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260806 00:46:16.560071 16840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:16.561568 16769 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260806 00:46:16.561702 16769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "e52e6eac8aae4758979bb30fb2a30ec1"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "dd062f518b64c4404ba3ca113a40952f"
server_key_iv: "7e4a4283eab9ab3aaa87ff6afcad715d"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:16.562263 16769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:16.577864 16769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:16.581532 16769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:16.581715 16769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:16.582396 16769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:16.583410 16769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:16.583480 16769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:16.583534 16769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:16.583564 16769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:16.594950 16769 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:44291
I20260806 00:46:16.596159 16769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:46:16.597326 17017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:44291 every 8 connection(s)
I20260806 00:46:16.604907 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 16769
I20260806 00:46:16.605408 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:50517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:45807
--local_ip_for_outbound_sockets=127.29.84.130
--tserver_master_addrs=127.29.84.190:35323
--webserver_port=37183
--webserver_interface=127.29.84.130
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.84.148:50517/kms with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:46:16 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:16.620685 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:16.599873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35323 (local address 127.29.84.129:40227)
0806 00:46:16.603418 (+ 3545us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:16.603473 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:16.604518 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:16.606667 (+ 2149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:16.606676 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:16.607008 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:16.607659 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:16.607680 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.609606 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.609613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:16.610696 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:16.610706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.612004 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.612008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:16.612199 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:16.612819 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:16.612839 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:16.614604 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:16.617167 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:16.617173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:16.617185 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:16.617516 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:16.617928 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:16.617931 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:16.617933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:16.618123 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:16.618864 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:16.618872 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:16.619195 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 00:46:16.619402 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 00:46:16.619698 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3321,"thread_start_us":155,"threads_started":1}
I20260806 00:46:16.622382 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:16.599735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:40227 (local address 127.29.84.190:35323)
0806 00:46:16.600192 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:16.600213 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:16.603638 (+ 3425us) server_negotiation.cc:407] Connection header received
0806 00:46:16.605162 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:16.605177 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:16.605824 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:16.606167 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:16.607858 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.609428 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.610821 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.611621 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.611783 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:16.614872 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:16.614878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:16.614891 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:16.614921 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:16.617027 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:16.617659 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:16.617665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:16.617671 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:16.617809 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:16.618244 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:16.618247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:16.618249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:16.618677 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:16.618802 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:16.621187 (+ 2385us) server_negotiation.cc:299] Negotiation successful
0806 00:46:16.621484 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":118,"threads_started":1}
I20260806 00:46:16.625653 17018 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35323
I20260806 00:46:16.625945 17018 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:16.626590 17018 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
W20260806 00:46:16.629006 17018 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:35323 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 00:46:16.629217 17018 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:16.629468 17018 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
I20260806 00:46:16.630504 17018 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:16.630749 17018 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
W20260806 00:46:16.631647 17018 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:46:16.758844 17022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:16.759138 17022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:16.759245 17022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:16.763768 17022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:16.763831 17022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:16.763903 17022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:46:16.770457 17022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.84.148:50517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:45807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=37183
--tserver_master_addrs=127.29.84.190:35323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:16.771477 17022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:16.772445 17022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:16.782485 17036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:16.782706 17033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:16.783248 17029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:16.783303 17022 server_base.cc:1034] running on GCE node
I20260806 00:46:16.783766 17022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:16.784451 17022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:16.785629 17022 hybrid_clock.cc:648] HybridClock initialized: now 1785977176785595 us; error 45 us; skew 500 ppm
Aug 06 00:46:16 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:16.789222 17022 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:46:16.790130 17022 webserver.cc:460] Webserver started at http://127.29.84.130:37183/ using document root <none> and password file <none>
I20260806 00:46:16.790747 17022 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:16.790844 17022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:46:16 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:16.808439 17022 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260806 00:46:16.811265 17050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:16.812602 17022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:46:16.812739 17022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "f0c578a02c6943c0a08538e50ce47fb2"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "85b51e1e7b3a33d7c1c6f55ea3c2e424"
server_key_iv: "ee956238db4b936b066f906ba664da6e"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:16.813271 17022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:16.830766 17022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:16.833745 17022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:16.833928 17022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:16.834491 17022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:16.835433 17022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:16.835498 17022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:16.835559 17022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:16.835597 17022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:16.846158 17022 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:45807
I20260806 00:46:16.846354 17229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:45807 every 8 connection(s)
I20260806 00:46:16.847368 17022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:46:16.851435 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 17022
I20260806 00:46:16.851923 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.84.148:50517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:45403
--local_ip_for_outbound_sockets=127.29.84.131
--tserver_master_addrs=127.29.84.190:35323
--webserver_port=33895
--webserver_interface=127.29.84.131
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.84.148:50517/kms with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:46:16 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:16.866544 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:16.849539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:50495 (local address 127.29.84.190:35323)
0806 00:46:16.849718 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:16.849723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:16.850959 (+ 1236us) server_negotiation.cc:407] Connection header received
0806 00:46:16.852609 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:16.852613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:16.852665 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:16.852717 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:16.854253 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.855112 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.856803 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.857184 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.857283 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:16.860246 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:16.860253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:16.860258 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:16.860293 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:16.862174 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:16.862885 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:16.862890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:16.862895 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:16.862990 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:16.863453 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:16.863458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:16.863461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:16.863721 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:16.863826 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:16.866191 (+ 2365us) server_negotiation.cc:299] Negotiation successful
0806 00:46:16.866335 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:46:16.867185 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:16.850253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35323 (local address 127.29.84.130:50495)
0806 00:46:16.850798 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:16.850856 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:16.852365 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:16.853117 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:16.853125 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:16.853440 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:16.854079 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:16.854099 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.855265 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.855269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:16.856660 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:16.856672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:16.857445 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:16.857449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:16.857682 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:16.858294 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:16.858315 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:16.860076 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:16.862329 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:16.862337 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:16.862353 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:16.862753 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:16.863134 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:16.863137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:16.863139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:16.863327 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:16.863888 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:16.863898 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:16.864219 (+ 321us) client_negotiation.cc:769] Sending connection context
0806 00:46:16.864422 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 00:46:16.864769 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":156,"threads_started":1}
I20260806 00:46:16.868402 17230 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35323
I20260806 00:46:16.868670 17230 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:16.869199 17230 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
W20260806 00:46:16.870690 17230 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:35323 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 00:46:16.870810 17230 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:16.870967 17230 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
I20260806 00:46:16.871701 17230 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:16.871858 17230 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
W20260806 00:46:16.872575 17230 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:46:16.987039 17233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:16.987371 17233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:16.987442 17233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:16.991632 17233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:16.991681 17233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:16.991734 17233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:46:16.997591 17233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.84.148:50517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:45403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=33895
--tserver_master_addrs=127.29.84.190:35323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:16.998719 17233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:16.999845 17233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:17.011719 17248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:17.011659 17246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:17.012305 17245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:17.012593 17233 server_base.cc:1034] running on GCE node
I20260806 00:46:17.012984 17233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:17.013691 17233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:17.014866 17233 hybrid_clock.cc:648] HybridClock initialized: now 1785977177014821 us; error 46 us; skew 500 ppm
Aug 06 00:46:17 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977177, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:17.021282 17233 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:46:17.022382 17233 webserver.cc:460] Webserver started at http://127.29.84.131:33895/ using document root <none> and password file <none>
I20260806 00:46:17.023175 17233 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:17.023334 17233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:46:17 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977177, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:17.046761 17233 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.000s
I20260806 00:46:17.049916 17264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:17.051522 17233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 00:46:17.051666 17233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "9067d6bf11f94fa18a1de8f372e0d610"
format_stamp: "Formatted at 2026-08-06 00:46:15 on dist-test-slave-n326"
server_key: "0179e4d89fa0710240ccb92155484404"
server_key_iv: "43ec74dbe187f13890fdf7efd366a8cd"
server_key_version: "kuduclusterkey@0"
I20260806 00:46:17.052363 17233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:17.073918 17233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:17.081796 17233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:17.082003 17233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:17.082683 17233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:17.083719 17233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:17.083786 17233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:17.083844 17233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:17.083881 17233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:17.094470 17233 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:45403
I20260806 00:46:17.096369 17233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:46:17.103259 17442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:45403 every 8 connection(s)
I20260806 00:46:17.106616 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 17233
Aug 06 00:46:17 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977177, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:17.114208 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:17.098842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:42895 (local address 127.29.84.190:35323)
0806 00:46:17.098968 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:17.098973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:17.100559 (+ 1586us) server_negotiation.cc:407] Connection header received
0806 00:46:17.101691 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:17.101695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:17.101749 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:17.101840 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:17.103259 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:17.104088 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:17.105817 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:17.106193 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:17.106273 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:17.110136 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:17.110139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:17.110142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:17.110167 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:17.111768 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:17.112362 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:17.112365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:17.112367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:17.112448 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:17.112855 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:17.112858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:17.112860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:17.113084 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:17.113173 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:17.113897 (+ 724us) server_negotiation.cc:299] Negotiation successful
0806 00:46:17.114060 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:46:17.115720 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:17.099327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:35323 (local address 127.29.84.131:42895)
0806 00:46:17.100381 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:17.100434 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:17.101444 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:17.102063 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:17.102072 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:17.102407 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:17.103084 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:17.103108 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:17.104251 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:17.104254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:17.105656 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:17.105667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:17.107142 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:17.107419 (+ 277us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:17.107617 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:17.108230 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:17.108250 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:17.109946 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:17.111919 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:17.111924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:17.111937 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:17.112251 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:17.112569 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:17.112572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:17.112574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:17.112738 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:17.113251 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:17.113257 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:17.113616 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 00:46:17.114436 (+ 820us) client_negotiation.cc:240] Negotiation successful
0806 00:46:17.114763 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":126,"threads_started":1}
I20260806 00:46:17.116915 17443 heartbeater.cc:347] Connected to a master server at 127.29.84.190:35323
I20260806 00:46:17.117147 17443 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:17.117640 17443 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
W20260806 00:46:17.119321 17443 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:35323 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 00:46:17.119445 17443 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:17.119602 17443 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
I20260806 00:46:17.120388 17443 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:17.120548 17443 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
W20260806 00:46:17.121405 17443 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:46:17.130179 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:17.107285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.29.84.190:35323)
0806 00:46:17.107993 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:17.108013 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:17.109232 (+ 1219us) server_negotiation.cc:407] Connection header received
0806 00:46:17.109272 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:17.109275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:17.109319 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:17.109399 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:17.120734 (+ 11335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:17.121941 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:17.123168 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:17.123499 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:17.125576 (+ 2077us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:17.125693 (+ 117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:17.125696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:17.125699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:17.125715 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:17.127491 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:17.128808 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:17.128812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:17.128813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:17.128905 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:17.129241 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:17.129245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:17.129246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:17.129451 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:17.129667 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:17.129895 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 00:46:17.130023 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":81,"threads_started":1}
Aug 06 00:46:17 dist-test-slave-n326 krb5kdc[15207](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977177, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:17.631907 17018 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:17.632187 17018 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
I20260806 00:46:17.633299 16723 ts_manager.cc:194] Registered new tserver with Master: e52e6eac8aae4758979bb30fb2a30ec1 (127.29.84.129:44291)
I20260806 00:46:17.634925 16723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:40227
Aug 06 00:46:17 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977177, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:17.872786 17230 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:17.873080 17230 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
I20260806 00:46:17.873834 16723 ts_manager.cc:194] Registered new tserver with Master: f0c578a02c6943c0a08538e50ce47fb2 (127.29.84.130:45807)
I20260806 00:46:17.874580 16723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:50495
Aug 06 00:46:18 dist-test-slave-n326 krb5kdc[15207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977177, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for HTTP/127.29.84.148@KRBTEST.COM
I20260806 00:46:18.121629 17443 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:18.121876 17443 heartbeater.cc:511] Master 127.29.84.190:35323 requested a full tablet report, sending...
I20260806 00:46:18.122610 16723 ts_manager.cc:194] Registered new tserver with Master: 9067d6bf11f94fa18a1de8f372e0d610 (127.29.84.131:45403)
I20260806 00:46:18.123371 16723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:42895
I20260806 00:46:18.126336 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:46:18.159186 17481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.135499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60606 (local address 127.29.84.190:35323)
0806 00:46:18.143239 (+ 7740us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:18.143244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:18.144392 (+ 1148us) server_negotiation.cc:407] Connection header received
0806 00:46:18.144552 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:18.144555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:18.144606 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:18.144693 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:18.145639 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.146993 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.148434 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.148817 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.148898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.150693 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:18.150698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:18.150701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:18.150725 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:18.152511 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:18.157504 (+ 4993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:18.157507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:18.157509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:18.157602 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:18.158063 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:18.158066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:18.158068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:18.158269 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:18.158377 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:18.158746 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 00:46:18.158981 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7650,"thread_start_us":101,"threads_started":1}
I20260806 00:46:18.163504 16723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:46:18.168370 16723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:46:18.217048 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.185352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49244 (local address 127.29.84.131:45403)
0806 00:46:18.185594 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:18.185601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:18.190588 (+ 4987us) server_negotiation.cc:407] Connection header received
0806 00:46:18.195328 (+ 4740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:18.195335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:18.195537 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:18.195664 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:18.196481 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.197658 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.215184 (+ 17526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.216091 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.216174 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.216318 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:18.216769 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 00:46:18.216877 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
I20260806 00:46:18.217693 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.184756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45403 (local address 127.0.0.1:49244)
0806 00:46:18.190488 (+ 5732us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:18.190502 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:18.195146 (+ 4644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:18.195918 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:18.195925 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:18.195949 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:18.196095 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.196102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.205838 (+ 9736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.205841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.207254 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.207266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.216419 (+ 9153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.216423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.216533 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.216573 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 00:46:18.217421 (+ 848us) client_negotiation.cc:240] Negotiation successful
0806 00:46:18.217554 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5670,"thread_start_us":90,"threads_started":1}
I20260806 00:46:18.220615 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.185012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48872 (local address 127.29.84.130:45807)
0806 00:46:18.185286 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:18.185293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:18.208065 (+ 22772us) server_negotiation.cc:407] Connection header received
0806 00:46:18.208316 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:18.208321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:18.208518 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:18.208635 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:18.209492 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.210656 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.218937 (+ 8281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.219734 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.219819 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.219954 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:18.220384 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 00:46:18.220479 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":102,"threads_started":1}
I20260806 00:46:18.220983 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.184756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:45807 (local address 127.0.0.1:48872)
0806 00:46:18.207484 (+ 22728us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:18.207499 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:18.208152 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:18.208880 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:18.208882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:18.208901 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:18.208990 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.208996 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.210803 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.210806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.212575 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.212586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.220084 (+ 7498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.220087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.220176 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.220199 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:46:18.220771 (+ 572us) client_negotiation.cc:240] Negotiation successful
0806 00:46:18.220842 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22663,"mutex_wait_us":821,"thread_start_us":111,"threads_started":1}
I20260806 00:46:18.221565 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.183373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:44291 (local address 127.0.0.1:45014)
0806 00:46:18.185941 (+ 2568us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:18.186002 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:18.186132 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:18.196249 (+ 10117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:18.196252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:18.196270 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:18.196359 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.196365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.212730 (+ 16365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.212734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.213925 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.213936 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.221172 (+ 7236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.221177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.221267 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.221288 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:46:18.221358 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 00:46:18.221449 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2397,"thread_start_us":108,"threads_started":1}
I20260806 00:46:18.221217 17333 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3ed32ead99486d8fb8611225c59d60 (DEFAULT_TABLE table=test-table [id=b07d9920408f402380b66a444f2d679e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:18.222247 17489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.183497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45014 (local address 127.29.84.129:44291)
0806 00:46:18.189911 (+ 6414us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:18.189917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:18.189944 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:46:18.190047 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:18.190052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:18.190259 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:18.197831 (+ 7572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:18.197990 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.211733 (+ 13743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.214140 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.214990 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.221833 (+ 6843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.221979 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:18.222028 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 00:46:18.222116 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6301,"thread_start_us":63,"threads_started":1}
I20260806 00:46:18.222378 17333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3ed32ead99486d8fb8611225c59d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:18.223735 17121 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3ed32ead99486d8fb8611225c59d60 (DEFAULT_TABLE table=test-table [id=b07d9920408f402380b66a444f2d679e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:18.224702 17121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3ed32ead99486d8fb8611225c59d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:18.229578 16912 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3ed32ead99486d8fb8611225c59d60 (DEFAULT_TABLE table=test-table [id=b07d9920408f402380b66a444f2d679e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:18.230592 16912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3ed32ead99486d8fb8611225c59d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:18.244678 17498 tablet_bootstrap.cc:492] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1: Bootstrap starting.
I20260806 00:46:18.247279 17498 tablet_bootstrap.cc:654] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:18.248157 17498 log.cc:824] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:18.250334 17498 tablet_bootstrap.cc:492] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1: No bootstrap required, opened a new log
I20260806 00:46:18.250561 17498 ts_tablet_manager.cc:1397] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:46:18.254810 17498 raft_consensus.cc:348] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } }
I20260806 00:46:18.255111 17498 raft_consensus.cc:374] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:18.255177 17498 raft_consensus.cc:729] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e52e6eac8aae4758979bb30fb2a30ec1, State: Initialized, Role: FOLLOWER
I20260806 00:46:18.255676 17498 consensus_queue.cc:260] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } }
I20260806 00:46:18.256692 17498 ts_tablet_manager.cc:1428] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260806 00:46:18.257043 17018 heartbeater.cc:503] Master 127.29.84.190:35323 was elected leader, sending a full tablet report...
I20260806 00:46:18.267448 17499 tablet_bootstrap.cc:492] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2: Bootstrap starting.
I20260806 00:46:18.269986 17499 tablet_bootstrap.cc:654] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:18.270897 17499 log.cc:824] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:18.275614 17501 tablet_bootstrap.cc:492] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: Bootstrap starting.
I20260806 00:46:18.278214 17501 tablet_bootstrap.cc:654] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:18.279142 17501 log.cc:824] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:18.280277 17499 tablet_bootstrap.cc:492] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2: No bootstrap required, opened a new log
I20260806 00:46:18.280539 17499 ts_tablet_manager.cc:1397] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260806 00:46:18.284878 17499 raft_consensus.cc:348] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } }
I20260806 00:46:18.285168 17499 raft_consensus.cc:374] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:18.285234 17499 raft_consensus.cc:729] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0c578a02c6943c0a08538e50ce47fb2, State: Initialized, Role: FOLLOWER
I20260806 00:46:18.285768 17499 consensus_queue.cc:260] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } }
I20260806 00:46:18.286799 17499 ts_tablet_manager.cc:1428] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:46:18.294718 17230 heartbeater.cc:503] Master 127.29.84.190:35323 was elected leader, sending a full tablet report...
I20260806 00:46:18.299044 17507 raft_consensus.cc:482] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:18.299212 17507 raft_consensus.cc:504] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } }
I20260806 00:46:18.302796 17507 leader_election.cc:288] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e52e6eac8aae4758979bb30fb2a30ec1 (127.29.84.129:44291), 9067d6bf11f94fa18a1de8f372e0d610 (127.29.84.131:45403)
I20260806 00:46:18.305418 17501 tablet_bootstrap.cc:492] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: No bootstrap required, opened a new log
I20260806 00:46:18.305650 17501 ts_tablet_manager.cc:1397] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260806 00:46:18.309836 17501 raft_consensus.cc:348] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } }
I20260806 00:46:18.310130 17501 raft_consensus.cc:374] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:18.310192 17501 raft_consensus.cc:729] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9067d6bf11f94fa18a1de8f372e0d610, State: Initialized, Role: FOLLOWER
I20260806 00:46:18.310703 17501 consensus_queue.cc:260] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } }
I20260806 00:46:18.311822 17501 ts_tablet_manager.cc:1428] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260806 00:46:18.312167 17443 heartbeater.cc:503] Master 127.29.84.190:35323 was elected leader, sending a full tablet report...
I20260806 00:46:18.318074 17489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.300923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:44575 (local address 127.29.84.129:44291)
0806 00:46:18.301079 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:18.301084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:18.303752 (+ 2668us) server_negotiation.cc:407] Connection header received
0806 00:46:18.303958 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:18.303961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:18.304011 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:18.304059 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:18.311172 (+ 7113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.311955 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.316680 (+ 4725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.317257 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.317352 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.317402 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:18.317845 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 00:46:18.317926 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:46:18.318531 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.300540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:44291 (local address 127.29.84.130:44575)
0806 00:46:18.303626 (+ 3086us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:18.303641 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:18.303832 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:18.304192 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:18.304195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:18.304210 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:18.304469 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.304476 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.312106 (+ 7630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.312111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.313375 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.313387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.317509 (+ 4122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.317513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.317683 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.317705 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:46:18.318299 (+ 594us) client_negotiation.cc:240] Negotiation successful
0806 00:46:18.318371 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3018,"thread_start_us":101,"threads_started":1}
I20260806 00:46:18.319133 16927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ed32ead99486d8fb8611225c59d60" candidate_uuid: "f0c578a02c6943c0a08538e50ce47fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" is_pre_election: true
I20260806 00:46:18.319422 16927 raft_consensus.cc:2454] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0c578a02c6943c0a08538e50ce47fb2 in term 0.
I20260806 00:46:18.319936 17055 leader_election.cc:302] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e52e6eac8aae4758979bb30fb2a30ec1, f0c578a02c6943c0a08538e50ce47fb2; no voters:
I20260806 00:46:18.320232 17507 raft_consensus.cc:2781] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:46:18.320322 17507 raft_consensus.cc:482] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:46:18.320458 17507 raft_consensus.cc:3037] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:18.321803 17507 raft_consensus.cc:504] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } }
I20260806 00:46:18.322162 17507 leader_election.cc:288] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [CANDIDATE]: Term 1 election: Requested vote from peers e52e6eac8aae4758979bb30fb2a30ec1 (127.29.84.129:44291), 9067d6bf11f94fa18a1de8f372e0d610 (127.29.84.131:45403)
I20260806 00:46:18.323215 16927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ed32ead99486d8fb8611225c59d60" candidate_uuid: "f0c578a02c6943c0a08538e50ce47fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52e6eac8aae4758979bb30fb2a30ec1"
I20260806 00:46:18.323343 16927 raft_consensus.cc:3037] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:18.324659 16927 raft_consensus.cc:2454] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0c578a02c6943c0a08538e50ce47fb2 in term 1.
I20260806 00:46:18.324963 17055 leader_election.cc:302] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e52e6eac8aae4758979bb30fb2a30ec1, f0c578a02c6943c0a08538e50ce47fb2; no voters:
I20260806 00:46:18.325177 17507 raft_consensus.cc:2781] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:46:18.325536 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.302652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:35775 (local address 127.29.84.131:45403)
0806 00:46:18.302763 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:18.302768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:18.313699 (+ 10931us) server_negotiation.cc:407] Connection header received
0806 00:46:18.313915 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:18.313918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:18.313959 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:18.314003 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:18.314392 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.315225 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.322324 (+ 7099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.322932 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.323044 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.323108 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:18.325337 (+ 2229us) server_negotiation.cc:299] Negotiation successful
0806 00:46:18.325418 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:46:18.325547 17511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.302300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:45403 (local address 127.29.84.130:35775)
0806 00:46:18.313622 (+ 11322us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:18.313635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:18.313802 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:18.314141 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:18.314144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:18.314159 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:18.314273 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.314279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.315361 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.315364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.316528 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:18.316539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.325009 (+ 8470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.325012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:18.325208 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.325232 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:46:18.325322 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 00:46:18.325391 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11249,"thread_start_us":92,"threads_started":1}
I20260806 00:46:18.326383 17507 raft_consensus.cc:686] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 1 LEADER]: Becoming Leader. State: Replica: f0c578a02c6943c0a08538e50ce47fb2, State: Running, Role: LEADER
I20260806 00:46:18.326426 17368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ed32ead99486d8fb8611225c59d60" candidate_uuid: "f0c578a02c6943c0a08538e50ce47fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9067d6bf11f94fa18a1de8f372e0d610" is_pre_election: true
I20260806 00:46:18.326669 17368 raft_consensus.cc:2454] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0c578a02c6943c0a08538e50ce47fb2 in term 0.
I20260806 00:46:18.326960 17369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ed32ead99486d8fb8611225c59d60" candidate_uuid: "f0c578a02c6943c0a08538e50ce47fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9067d6bf11f94fa18a1de8f372e0d610"
I20260806 00:46:18.327088 17369 raft_consensus.cc:3037] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:18.327469 17507 consensus_queue.cc:237] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } }
I20260806 00:46:18.328269 17369 raft_consensus.cc:2454] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0c578a02c6943c0a08538e50ce47fb2 in term 1.
I20260806 00:46:18.331182 16723 catalog_manager.cc:5495] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0c578a02c6943c0a08538e50ce47fb2 (127.29.84.130). New cstate: current_term: 1 leader_uuid: "f0c578a02c6943c0a08538e50ce47fb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0c578a02c6943c0a08538e50ce47fb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45807 } health_report { overall_health: HEALTHY } } }
W20260806 00:46:18.350404 17231 tablet.cc:2369] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:46:18.353734 17019 tablet.cc:2369] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:46:18.355165 17444 tablet.cc:2369] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:46:18.379027 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.374386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.29.84.130:45807)
0806 00:46:18.374518 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:18.374523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:18.374538 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:46:18.374618 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:18.374620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:18.374685 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:18.374830 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:46:18.375277 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.376071 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.377271 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:18.377616 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:18.377718 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:18.378047 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:46:18.378544 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:46:18.378659 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:18.378788 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 00:46:18.378846 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:46:18.385671 17369 raft_consensus.cc:1261] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [term 1 FOLLOWER]: Refusing update from remote peer f0c578a02c6943c0a08538e50ce47fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:18.385826 16927 raft_consensus.cc:1261] T 0f3ed32ead99486d8fb8611225c59d60 P e52e6eac8aae4758979bb30fb2a30ec1 [term 1 FOLLOWER]: Refusing update from remote peer f0c578a02c6943c0a08538e50ce47fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:18.386423 17507 consensus_queue.cc:1035] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9067d6bf11f94fa18a1de8f372e0d610" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 45403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:46:18.386587 17506 consensus_queue.cc:1035] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e52e6eac8aae4758979bb30fb2a30ec1" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:46:18.397176 17528 mvcc.cc:205] Tried to move back new op lower bound from 7315362522660618240 to 7315362522433605632. Current Snapshot: MvccSnapshot[applied={T|T < 7315362522660618240}]
I20260806 00:46:18.397502 17529 mvcc.cc:205] Tried to move back new op lower bound from 7315362522660618240 to 7315362522433605632. Current Snapshot: MvccSnapshot[applied={T|T < 7315362522660618240}]
I20260806 00:46:18.399210 17530 mvcc.cc:205] Tried to move back new op lower bound from 7315362522660618240 to 7315362522433605632. Current Snapshot: MvccSnapshot[applied={T|T < 7315362522660618240}]
I20260806 00:46:18.407536 16722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:46:18.407702 16722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:46:18.410284 16722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ee08a50cc24418e804499b4cea3f3ef: Sending DeleteTablet for 3 replicas of tablet 0f3ed32ead99486d8fb8611225c59d60
I20260806 00:46:18.411479 17333 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3ed32ead99486d8fb8611225c59d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:18 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:49244
I20260806 00:46:18.411921 17121 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3ed32ead99486d8fb8611225c59d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:18 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:48872
I20260806 00:46:18.412470 16912 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3ed32ead99486d8fb8611225c59d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:18 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:45014
I20260806 00:46:18.413895 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 16769
I20260806 00:46:18.415256 17545 tablet_replica.cc:330] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: stopping tablet replica
I20260806 00:46:18.415560 17545 raft_consensus.cc:2229] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:46:18.415865 17545 raft_consensus.cc:2258] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:46:18.419083 17546 tablet_replica.cc:330] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2: stopping tablet replica
I20260806 00:46:18.419448 17546 raft_consensus.cc:2229] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:46:18.419852 17546 raft_consensus.cc:2258] T 0f3ed32ead99486d8fb8611225c59d60 P f0c578a02c6943c0a08538e50ce47fb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:46:18.431380 17545 ts_tablet_manager.cc:1905] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 00:46:18.431419 16702 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:44291 (error 108)
I20260806 00:46:18.432197 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 17022
I20260806 00:46:18.434657 17545 ts_tablet_manager.cc:1918] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:46:18.434762 17545 log.cc:1196] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/0f3ed32ead99486d8fb8611225c59d60
I20260806 00:46:18.435112 17545 ts_tablet_manager.cc:1939] T 0f3ed32ead99486d8fb8611225c59d60 P 9067d6bf11f94fa18a1de8f372e0d610: Deleting consensus metadata
I20260806 00:46:18.439812 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.439450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:44291 (local address 127.0.0.1:45028)
0806 00:46:18.439613 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:18.439701 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:44291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 00:46:18.440037 16702 catalog_manager.cc:4584] TS e52e6eac8aae4758979bb30fb2a30ec1 (127.29.84.129:44291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f3ed32ead99486d8fb8611225c59d60: Network error: Client connection negotiation failed: client connection to 127.29.84.129:44291: connect: Connection refused (error 111)
I20260806 00:46:18.440431 16702 catalog_manager.cc:4841] TS 9067d6bf11f94fa18a1de8f372e0d610 (127.29.84.131:45403): tablet 0f3ed32ead99486d8fb8611225c59d60 (table test-table [id=b07d9920408f402380b66a444f2d679e]) successfully deleted
I20260806 00:46:18.445384 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 17233
I20260806 00:46:18.447175 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:18.445564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:45807 (local address 127.0.0.1:48894)
0806 00:46:18.447004 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:18.447084 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.130:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1351}
I20260806 00:46:18.454159 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 16672
I20260806 00:46:18.467375 30034 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 00:46:18.813632 30034 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 00:46:18.813733 30034 mini_ranger.cc:66] Stopping Ranger...
I20260806 00:46:19.158092 30034 mini_ranger.cc:68] Stopped Ranger
2026-08-06 00:46:19.158 UTC [15247] LOG: received smart shutdown request
2026-08-06 00:46:19.162 UTC [15247] LOG: background worker "logical replication launcher" (PID 15258) exited with exit code 1
2026-08-06 00:46:19.163 UTC [15253] LOG: shutting down
2026-08-06 00:46:19.316 UTC [15247] LOG: database system is shut down
2026-08-06T00:46:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46585 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:46:19 dist-test-slave-n326 krb5kdc[17568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:46:19 dist-test-slave-n326 krb5kdc[17568](info): set up 2 sockets
Aug 06 00:46:19 dist-test-slave-n326 krb5kdc[17568](info): commencing operation
krb5kdc: starting...
W20260806 00:46:20.654073 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:46:20 dist-test-slave-n326 krb5kdc[17568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:46:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:46:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:46:20.774413 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:42561
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:42561,127.29.84.189:33519,127.29.84.188:33291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:20.881289 17586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:20.881529 17586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:20.881567 17586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:20.885275 17586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:20.885322 17586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:20.885341 17586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:20.885357 17586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:20.885376 17586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:20.890144 17586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:42561,127.29.84.189:33519,127.29.84.188:33291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:42561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:20.890993 17586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:20.891932 17586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:20.899780 17597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:20.899780 17592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:20.899809 17595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:20.900120 17586 server_base.cc:1034] running on GCE node
I20260806 00:46:20.900710 17586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:20.901746 17586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:20.902909 17586 hybrid_clock.cc:648] HybridClock initialized: now 1785977180902899 us; error 26 us; skew 500 ppm
Aug 06 00:46:20 dist-test-slave-n326 krb5kdc[17568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977180, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:20.905503 17586 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:46:20.906332 17586 webserver.cc:460] Webserver started at http://127.29.84.190:40415/ using document root <none> and password file <none>
I20260806 00:46:20.906831 17586 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:20.906888 17586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:20.907159 17586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:20.908989 17586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "74f9b3805c1e480682f1e4034e2702d7"
format_stamp: "Formatted at 2026-08-06 00:46:20 on dist-test-slave-n326"
server_key: "470203491c8b690cfc83b904e30b280d"
server_key_iv: "30f5fb3098ebc672ac9bae915bbb1104"
server_key_version: "encryptionkey@0"
I20260806 00:46:20.909543 17586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "74f9b3805c1e480682f1e4034e2702d7"
format_stamp: "Formatted at 2026-08-06 00:46:20 on dist-test-slave-n326"
server_key: "470203491c8b690cfc83b904e30b280d"
server_key_iv: "30f5fb3098ebc672ac9bae915bbb1104"
server_key_version: "encryptionkey@0"
I20260806 00:46:20.912987 17586 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:46:20.915179 17604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:20.916252 17586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:20.916388 17586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "74f9b3805c1e480682f1e4034e2702d7"
format_stamp: "Formatted at 2026-08-06 00:46:20 on dist-test-slave-n326"
server_key: "470203491c8b690cfc83b904e30b280d"
server_key_iv: "30f5fb3098ebc672ac9bae915bbb1104"
server_key_version: "encryptionkey@0"
I20260806 00:46:20.916517 17586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:20.942817 17586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:20.945493 17586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:20.945703 17586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:20.952831 17586 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:42561
I20260806 00:46:20.952859 17689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:42561 every 8 connection(s)
I20260806 00:46:20.953670 17586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:46:20.956212 17690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:20.960104 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 17586
I20260806 00:46:20.960265 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:46:20.960005 17690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:46:20.960534 30034 external_mini_cluster.cc:1428] Setting key 6d28296336a14326d6a9932ec9210227
WARNING: no policy specified for kudu/127.29.84.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.189@KRBTEST.COM" created.
Aug 06 00:46:20 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977180, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:20.978731 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:20.962210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59380 (local address 127.29.84.190:42561)
0806 00:46:20.962751 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:20.962768 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:20.963117 (+ 349us) server_negotiation.cc:407] Connection header received
0806 00:46:20.964365 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:20.964369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:20.964547 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:20.964683 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:20.965799 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:20.966626 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:20.968072 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:20.968399 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:20.968508 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:20.973791 (+ 5283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:20.973796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:20.973804 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:20.973825 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:20.975765 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:20.976423 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:20.976427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:20.976429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:20.976545 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:20.976994 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:20.976997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:20.977000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:20.977332 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:20.977430 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:20.977898 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 00:46:20.978067 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"spinlock_wait_cycles":47360,"thread_start_us":160,"threads_started":1}
I20260806 00:46:20.978734 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:20.962210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:42561 (local address 127.0.0.1:59380)
0806 00:46:20.962907 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:20.962963 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:20.964090 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:20.964741 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:20.964749 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:20.965189 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:20.965619 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:20.965643 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:20.966759 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:20.966765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:20.967935 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:20.967948 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:20.968567 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:20.968571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:20.968714 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:20.969246 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:20.969273 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:20.973639 (+ 4366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:20.975908 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:20.975920 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:20.975931 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:20.976295 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:20.976651 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:20.976658 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:20.976663 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:20.976844 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:20.977459 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:20.977468 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:20.977638 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 00:46:20.977826 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 00:46:20.978200 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":51,"thread_start_us":145,"threads_started":1}
I20260806 00:46:20.980976 17690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:46:20.981804 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:20.981388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:33519 (local address 127.0.0.1:39034)
0806 00:46:20.981632 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:20.981700 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:46:20.981993 17605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
I20260806 00:46:20.982738 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:20.982562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:33519 (local address 127.0.0.1:39050)
0806 00:46:20.982652 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:20.982693 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 00:46:20.982890 17690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.189:33519: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab.
Entry for principal kudu/127.29.84.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab.
I20260806 00:46:20.988274 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.189:33519
--webserver_interface=127.29.84.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab
--principal=kudu/127.29.84.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:42561,127.29.84.189:33519,127.29.84.188:33291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:46:21.032121 17690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } attempt: 1
I20260806 00:46:21.032899 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.032619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:33519 (local address 127.0.0.1:39066)
0806 00:46:21.032782 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.032835 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 00:46:21.033593 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.033378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:33519 (local address 127.0.0.1:39070)
0806 00:46:21.033481 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.033543 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:46:21.033720 17690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.189:33519: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
W20260806 00:46:21.096268 17699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:21.096509 17699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:21.096593 17699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:21.100407 17699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:21.100483 17699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:21.100519 17699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:21.100574 17699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:21.100610 17699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:21.101874 17690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } attempt: 2
I20260806 00:46:21.102558 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.102295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:33519 (local address 127.0.0.1:39076)
0806 00:46:21.102434 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.102495 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 00:46:21.103359 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.103137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:33519 (local address 127.0.0.1:39090)
0806 00:46:21.103250 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.103301 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 00:46:21.103518 17690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.189:33519: Network error: Client connection negotiation failed: client connection to 127.29.84.189:33519: connect: Connection refused (error 111)
I20260806 00:46:21.105880 17699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:42561,127.29.84.189:33519,127.29.84.188:33291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab
--principal=kudu/127.29.84.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.189:33519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.84.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:21.106726 17699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:21.107694 17699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:21.115302 17711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:21.115298 17706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:21.115334 17709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:21.115540 17699 server_base.cc:1034] running on GCE node
I20260806 00:46:21.116129 17699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:21.117100 17699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:21.118249 17699 hybrid_clock.cc:648] HybridClock initialized: now 1785977181118239 us; error 27 us; skew 500 ppm
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977181, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:21.121088 17699 init.cc:377] Logged in from keytab as kudu/127.29.84.189@KRBTEST.COM (short username kudu)
I20260806 00:46:21.121904 17699 webserver.cc:460] Webserver started at http://127.29.84.189:33807/ using document root <none> and password file <none>
I20260806 00:46:21.122414 17699 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:21.122474 17699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:21.122709 17699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:21.124675 17699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/data/instance:
uuid: "a5305f8aa0004ed3a1cd91dbda3448a4"
format_stamp: "Formatted at 2026-08-06 00:46:21 on dist-test-slave-n326"
server_key: "0989b1d300e9c92bd6f617d7148ffb80"
server_key_iv: "4cf8afd4ea8b29508043412dcb28dd04"
server_key_version: "encryptionkey@0"
I20260806 00:46:21.125195 17699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/wal/instance:
uuid: "a5305f8aa0004ed3a1cd91dbda3448a4"
format_stamp: "Formatted at 2026-08-06 00:46:21 on dist-test-slave-n326"
server_key: "0989b1d300e9c92bd6f617d7148ffb80"
server_key_iv: "4cf8afd4ea8b29508043412dcb28dd04"
server_key_version: "encryptionkey@0"
I20260806 00:46:21.128515 17699 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:46:21.130645 17720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:21.131752 17699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:21.131896 17699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/wal
uuid: "a5305f8aa0004ed3a1cd91dbda3448a4"
format_stamp: "Formatted at 2026-08-06 00:46:21 on dist-test-slave-n326"
server_key: "0989b1d300e9c92bd6f617d7148ffb80"
server_key_iv: "4cf8afd4ea8b29508043412dcb28dd04"
server_key_version: "encryptionkey@0"
I20260806 00:46:21.132009 17699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:21.144631 17699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:21.147269 17699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:21.147449 17699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:21.154202 17699 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.189:33519
I20260806 00:46:21.154222 17797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.189:33519 every 8 connection(s)
I20260806 00:46:21.155053 17699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
I20260806 00:46:21.157536 17799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:21.160929 17799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:46:21.163830 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 17699
I20260806 00:46:21.163954 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-1/wal/instance
I20260806 00:46:21.164233 30034 external_mini_cluster.cc:1428] Setting key 23a39bf92ac3e301fcdc3dfd3ea5d1aa
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977181, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.84.188@KRBTEST.COM; defaulting to no policy
I20260806 00:46:21.175237 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.162876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:42561 (local address 127.0.0.1:59392)
0806 00:46:21.163346 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.163384 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:21.164199 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:21.164744 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:21.164751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:21.165050 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:21.165482 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.165494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.166530 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.166534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.167482 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.167494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.168198 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.168200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.168329 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.168939 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:21.168967 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:21.170805 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:21.172643 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.172651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.172665 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.173026 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.173281 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.173284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.173286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.173491 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.173862 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:21.173867 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:21.174074 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 00:46:21.174239 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 00:46:21.174500 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":179,"threads_started":1}
I20260806 00:46:21.175315 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.162573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.29.84.190:42561)
0806 00:46:21.162698 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.162702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.163483 (+ 781us) server_negotiation.cc:407] Connection header received
0806 00:46:21.164422 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.164425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.164486 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.164586 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.165634 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.166385 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.167626 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.168052 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.168180 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.170999 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.171002 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.171004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.171027 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.172505 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.173121 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.173123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.173125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.173195 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.173587 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.173590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.173591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.173758 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.173837 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.175049 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.175206 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.188@KRBTEST.COM" created.
I20260806 00:46:21.176733 17799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977181, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:46:21.186018 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.177197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:33519 (local address 127.0.0.1:39098)
0806 00:46:21.177314 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.177330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:21.177404 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:21.178032 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:21.178034 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:21.178165 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:21.178248 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.178253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.179225 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.179231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.179737 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.179744 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.180307 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.180310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.180401 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.180897 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:21.180907 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:21.182053 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:21.184491 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.184493 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.184494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.184686 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.184999 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.185001 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.185002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.185072 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.185606 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:21.185607 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:21.185680 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 00:46:21.185759 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 00:46:21.185851 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 00:46:21.186168 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.177323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.29.84.189:33519)
0806 00:46:21.177618 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.177632 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.177658 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:46:21.177707 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.177714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.177901 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.178055 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.178359 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.179122 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.179850 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.180199 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.180312 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.182199 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.182212 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.182228 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.182261 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.184378 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.184801 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.184810 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.184813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.184904 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.185171 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.185176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.185178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.185508 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.185617 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.185832 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.185977 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":121,"threads_started":1}
I20260806 00:46:21.188149 17799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:46:21.188822 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.188529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51802)
0806 00:46:21.188633 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.188698 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 00:46:21.189057 17724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
I20260806 00:46:21.189885 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.189668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51816)
0806 00:46:21.189784 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.189826 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:46:21.190063 17799 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:33291: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab.
Entry for principal kudu/127.29.84.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab.
I20260806 00:46:21.193918 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.188:33291
--webserver_interface=127.29.84.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab
--principal=kudu/127.29.84.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:42561,127.29.84.189:33519,127.29.84.188:33291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:46:21.194693 17690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } attempt: 3
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977180, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:46:21.203043 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.195154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:33519 (local address 127.0.0.1:39110)
0806 00:46:21.195286 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.195304 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:21.195371 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:21.195657 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:21.195659 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:21.195796 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:21.195896 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.195901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.196930 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.196932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.197501 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.197510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.197985 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.197989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.198107 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.198624 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:21.198636 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:21.199892 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:21.201615 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.201618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.201619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.201816 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.202088 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.202092 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.202094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.202181 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.202535 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:21.202538 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:21.202654 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 00:46:21.202758 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:46:21.202893 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 00:46:21.203037 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.195225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39110 (local address 127.29.84.189:33519)
0806 00:46:21.195480 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.195483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.195495 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:46:21.195521 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.195523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.195553 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.195611 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.196014 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.196824 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.197638 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.197868 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.197972 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.200025 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.200029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.200031 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.200058 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.201489 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.201918 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.201922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.201924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.201997 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.202273 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.202278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.202280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.202441 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.202540 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.202808 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.202908 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260806 00:46:21.204049 17690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:46:21.204723 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.204500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51830)
0806 00:46:21.204614 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.204663 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 00:46:21.205528 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.205308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51840)
0806 00:46:21.205443 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.205483 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260806 00:46:21.205724 17690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:33291: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
I20260806 00:46:21.236872 17690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } attempt: 1
I20260806 00:46:21.237439 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.237220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51856)
0806 00:46:21.237335 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.237387 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 00:46:21.238085 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.237898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51868)
0806 00:46:21.237992 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.238036 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 00:46:21.238240 17690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:33291: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
I20260806 00:46:21.239283 17799 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } attempt: 1
I20260806 00:46:21.239843 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.239640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51876)
0806 00:46:21.239748 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.239793 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 00:46:21.240459 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.240266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51892)
0806 00:46:21.240367 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.240414 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:46:21.240605 17799 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:33291: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
W20260806 00:46:21.304733 17809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:21.305009 17809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:21.305076 17809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 00:46:21.308840 17799 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } attempt: 2
W20260806 00:46:21.309195 17809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:21.309266 17809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:21.309331 17809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:21.309384 17809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:21.309422 17809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:21.309723 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.309407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51898)
0806 00:46:21.309566 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.309635 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 00:46:21.310464 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.310240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51910)
0806 00:46:21.310351 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.310404 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 00:46:21.310612 17799 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:33291: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
I20260806 00:46:21.313391 17690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } attempt: 2
I20260806 00:46:21.314114 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.313850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51918)
0806 00:46:21.313998 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.314049 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 00:46:21.314739 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.314558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51924)
0806 00:46:21.314650 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.314691 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 00:46:21.314917 17690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:33291: Network error: Client connection negotiation failed: client connection to 127.29.84.188:33291: connect: Connection refused (error 111)
I20260806 00:46:21.315627 17809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:42561,127.29.84.189:33519,127.29.84.188:33291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab
--principal=kudu/127.29.84.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.188:33291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.84.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:21.316605 17809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:21.317549 17809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:21.325855 17819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:21.325843 17816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:21.325834 17817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:21.326217 17809 server_base.cc:1034] running on GCE node
I20260806 00:46:21.326771 17809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:21.327821 17809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:21.329005 17809 hybrid_clock.cc:648] HybridClock initialized: now 1785977181328976 us; error 49 us; skew 500 ppm
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977181, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:21.331651 17809 init.cc:377] Logged in from keytab as kudu/127.29.84.188@KRBTEST.COM (short username kudu)
I20260806 00:46:21.332388 17809 webserver.cc:460] Webserver started at http://127.29.84.188:41389/ using document root <none> and password file <none>
I20260806 00:46:21.332835 17809 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:21.332888 17809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:21.333051 17809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:21.334960 17809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/data/instance:
uuid: "f69a71371b2e49e79814aad2847a2222"
format_stamp: "Formatted at 2026-08-06 00:46:21 on dist-test-slave-n326"
server_key: "fbb248e9ff74a3b3f69f5fb3fa3f899f"
server_key_iv: "b58704a0734721f76816203476a478e8"
server_key_version: "encryptionkey@0"
I20260806 00:46:21.335585 17809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/wal/instance:
uuid: "f69a71371b2e49e79814aad2847a2222"
format_stamp: "Formatted at 2026-08-06 00:46:21 on dist-test-slave-n326"
server_key: "fbb248e9ff74a3b3f69f5fb3fa3f899f"
server_key_iv: "b58704a0734721f76816203476a478e8"
server_key_version: "encryptionkey@0"
I20260806 00:46:21.338886 17809 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:46:21.341147 17834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:21.342242 17809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:21.342365 17809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/wal
uuid: "f69a71371b2e49e79814aad2847a2222"
format_stamp: "Formatted at 2026-08-06 00:46:21 on dist-test-slave-n326"
server_key: "fbb248e9ff74a3b3f69f5fb3fa3f899f"
server_key_iv: "b58704a0734721f76816203476a478e8"
server_key_version: "encryptionkey@0"
I20260806 00:46:21.342465 17809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:21.366350 17809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:21.368880 17809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:21.369086 17809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:21.375980 17809 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.188:33291
I20260806 00:46:21.375998 17918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.188:33291 every 8 connection(s)
I20260806 00:46:21.376885 17809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
I20260806 00:46:21.379392 17919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:21.380838 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 17809
I20260806 00:46:21.380967 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977020829091-30034-0/minicluster-data/master-2/wal/instance
I20260806 00:46:21.381254 30034 external_mini_cluster.cc:1428] Setting key d19862c3d55e8999dcb57599d015a3b5
I20260806 00:46:21.383612 17919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977181, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:21.394308 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.385540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59394 (local address 127.29.84.190:42561)
0806 00:46:21.385733 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.385737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.386050 (+ 313us) server_negotiation.cc:407] Connection header received
0806 00:46:21.386210 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.386213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.386265 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.386354 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.386972 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.387529 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.388440 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.388793 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.388931 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.391103 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.391107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.391110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.391138 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.392443 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.392963 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.392966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.392967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.393038 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.393387 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.393389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.393391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.393549 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.393629 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.394017 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.394148 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:46:21.395000 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.382487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51936 (local address 127.29.84.188:33291)
0806 00:46:21.382927 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.382942 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.382975 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:46:21.383616 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.383632 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.383935 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.384285 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.385163 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.386507 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.387437 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.387915 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.388084 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.390614 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.390623 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.390637 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.390662 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.392411 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.392878 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.392886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.392890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.393016 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.393294 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.393297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.393299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.393693 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.393870 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.394071 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.394264 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":171,"threads_started":1}
I20260806 00:46:21.395401 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.385468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:42561 (local address 127.0.0.1:59394)
0806 00:46:21.385931 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.385967 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:21.386108 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:21.386404 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:21.386410 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:21.386729 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:21.386847 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.386853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.387679 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.387682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.388289 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.388298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.388966 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.388969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.389058 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.389553 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:21.389568 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:21.390938 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:21.392609 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.392616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.392623 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.392866 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.393162 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.393164 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.393165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.393293 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.393650 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:21.393656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:21.393777 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 00:46:21.393935 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 00:46:21.395215 (+ 1280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":981,"thread_start_us":150,"threads_started":1}
I20260806 00:46:21.396464 17919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977181, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:46:21.401767 17799 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } attempt: 3
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:21.404671 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.396887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:33519 (local address 127.0.0.1:39122)
0806 00:46:21.396971 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.396985 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:21.397058 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:21.397354 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:21.397357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:21.397553 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:21.397657 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.397667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.398444 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.398447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.398906 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.398934 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.399446 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.399450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.399541 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.399970 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:21.399981 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:21.401234 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:21.403160 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.403162 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.403164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.403393 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.403696 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.403700 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.403702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.403792 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.404222 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:21.404224 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:21.404305 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 00:46:21.404409 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:46:21.404533 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 00:46:21.405099 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.396981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39122 (local address 127.29.84.189:33519)
0806 00:46:21.397142 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.397146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.397159 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:46:21.397203 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.397205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.397242 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.397332 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.397777 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.398335 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.399063 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.399334 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.399412 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.401362 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.401365 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.401367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.401401 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.403020 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.403476 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.403481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.403483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.403567 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.403900 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.403906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.403909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.404122 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.404236 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.404485 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.404606 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:46:21.405843 17919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:46:21.406433 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.397360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59404 (local address 127.29.84.190:42561)
0806 00:46:21.397490 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.397492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.397543 (+ 51us) server_negotiation.cc:407] Connection header received
0806 00:46:21.397667 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.397669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.397700 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.397796 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.398350 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.398872 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.399874 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.400118 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.400196 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.402724 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.402728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.402730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.402754 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.404519 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.404990 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.404994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.404996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.405077 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.405430 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.405435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.405437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.405675 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.405894 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.406182 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.406287 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977181, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977181, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
I20260806 00:46:21.411960 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.402144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51946)
0806 00:46:21.402503 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.402515 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:21.402572 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:21.403001 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:21.403004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:21.403208 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:21.403304 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.403311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.404165 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.404168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.404905 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.404915 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.406044 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.406047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.406132 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.406948 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:21.406959 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:21.408384 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:21.410288 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.410291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.410294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.410525 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.410835 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.410838 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.410839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.410917 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.411448 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:21.411451 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:21.411565 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 00:46:21.411687 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 00:46:21.411802 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":17}
I20260806 00:46:21.412561 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.402213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51946 (local address 127.29.84.188:33291)
0806 00:46:21.402780 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.402786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.402800 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:46:21.402830 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.402833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.402872 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.403248 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.403407 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.404063 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.405039 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.405908 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.406039 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.408539 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.408543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.408545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.408566 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.410186 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.410649 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.410652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.410654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.410732 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.411038 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.411042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.411043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.411261 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.411433 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.412248 (+ 815us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.412428 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500}
I20260806 00:46:21.414114 17690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } attempt: 3
I20260806 00:46:21.415326 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.406464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51956)
0806 00:46:21.406563 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.406577 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:21.406643 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:21.407237 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:21.407238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:21.407377 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:21.407460 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.407467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.408284 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.408290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.409063 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.409074 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.409704 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.409707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.409836 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.410276 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:21.410291 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:21.411624 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:21.413659 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.413661 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.413663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.413838 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.414135 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.414139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.414140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.414289 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.414833 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:21.414836 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:21.414913 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 00:46:21.415015 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:46:21.415153 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":18}
I20260806 00:46:21.415436 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.406636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.29.84.188:33291)
0806 00:46:21.406939 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.406948 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.406960 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:46:21.406993 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.406996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.407032 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.407150 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.407587 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.408180 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.409199 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.409574 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.409705 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.411762 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.411768 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.411772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.411798 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.413545 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.413937 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.413940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.413942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.414024 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.414370 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.414376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.414378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.414715 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.414801 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.415122 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.415233 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":221,"thread_start_us":110,"threads_started":1}
I20260806 00:46:21.415783 17799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4: Bootstrap starting.
I20260806 00:46:21.417829 17799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4: Neither blocks nor log segments found. Creating new log.
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977180, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
I20260806 00:46:21.418713 17799 log.cc:824] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:46:21 dist-test-slave-n326 krb5kdc[17568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:46:21.421207 17799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4: No bootstrap required, opened a new log
I20260806 00:46:21.421617 17919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222: Bootstrap starting.
I20260806 00:46:21.423704 17919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:21.424305 17919 log.cc:824] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:21.424448 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.408169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39124 (local address 127.29.84.189:33519)
0806 00:46:21.408261 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.408265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.408437 (+ 172us) server_negotiation.cc:407] Connection header received
0806 00:46:21.408533 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.408536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.408577 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.408657 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.409418 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.410166 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.411125 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.411488 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.411582 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.420961 (+ 9379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.420964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.420966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.420986 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.422560 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.423037 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.423040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.423042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.423144 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.423585 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.423588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.423590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.423767 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.423899 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.424225 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.424324 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":38}
I20260806 00:46:21.425628 17799 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } }
I20260806 00:46:21.425820 17799 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:21.425880 17799 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5305f8aa0004ed3a1cd91dbda3448a4, State: Initialized, Role: FOLLOWER
I20260806 00:46:21.426323 17799 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } }
I20260806 00:46:21.427304 17930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.427440 17930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:46:21.427462 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.414485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:33291 (local address 127.0.0.1:51972)
0806 00:46:21.414600 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:21.414614 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:21.414684 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:21.415012 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:21.415014 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:21.415214 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:21.415308 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.415314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.416364 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.416367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.416828 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:21.416836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.417235 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.417238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:21.417315 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.417703 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:21.417716 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:21.418983 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:21.425963 (+ 6980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.425966 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.425968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.426197 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.426462 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:21.426465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:21.426466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:21.426567 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:21.426987 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:21.426990 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:21.427105 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 00:46:21.427209 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:46:21.427327 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 00:46:21.427702 17799 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:21.427847 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:46:21.428193 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.414505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51972 (local address 127.29.84.188:33291)
0806 00:46:21.414814 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.414820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.414831 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:46:21.414855 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.414857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.414890 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.415715 (+ 825us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.415783 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.416251 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.416936 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.417130 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.417223 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.419209 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.419212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.419217 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.419247 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.425872 (+ 6625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.426278 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.426281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.426283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.426361 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.426665 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.426669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.426670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.426878 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.426980 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.427890 (+ 910us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.428053 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249}
I20260806 00:46:21.431056 17690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7: Bootstrap starting.
I20260806 00:46:21.431682 17919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222: No bootstrap required, opened a new log
I20260806 00:46:21.433583 17690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:21.434343 17690 log.cc:824] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:21.435798 17919 raft_consensus.cc:348] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } }
I20260806 00:46:21.436029 17919 raft_consensus.cc:374] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:21.436077 17919 raft_consensus.cc:729] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69a71371b2e49e79814aad2847a2222, State: Initialized, Role: FOLLOWER
I20260806 00:46:21.436450 17690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7: No bootstrap required, opened a new log
I20260806 00:46:21.436501 17919 consensus_queue.cc:260] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } }
I20260806 00:46:21.437744 17919 sys_catalog.cc:564] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:21.435026 17699 master_runner.cc:186] Error getting master registration for 127.29.84.190:42561: OK, instance_id { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" instance_seqno: 1785977180950804 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:46:21.439798 17699 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 00:46:21.442725 17980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:46:21.442806 17980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 00:46:21.443046 17957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:46:21.443156 17957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:46:21.441133 17809 master_runner.cc:186] Error getting master registration for 127.29.84.190:42561: OK, instance_id { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" instance_seqno: 1785977180950804 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:46:21.445359 17809 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:46:21.446024 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.430727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59406 (local address 127.29.84.190:42561)
0806 00:46:21.430862 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.430866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.431169 (+ 303us) server_negotiation.cc:407] Connection header received
0806 00:46:21.431337 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.431339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.431378 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.431422 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.433418 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.434064 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.437041 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.437319 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.437388 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.440426 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.440429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.440449 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.440478 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.442183 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.444687 (+ 2504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.444691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.444694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.444777 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.445130 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.445134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.445135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.445337 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.445449 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.445783 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.445893 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 00:46:21.446247 17690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } }
I20260806 00:46:21.446444 17690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:21.446514 17690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74f9b3805c1e480682f1e4034e2702d7, State: Initialized, Role: FOLLOWER
I20260806 00:46:21.446923 17690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } }
I20260806 00:46:21.448185 17690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:21.449528 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.430763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.29.84.189:33519)
0806 00:46:21.430930 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.430934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.432413 (+ 1479us) server_negotiation.cc:407] Connection header received
0806 00:46:21.432529 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.432532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.432567 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.432612 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.434967 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.435670 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.437850 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.438159 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.438239 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.442933 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.442936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.442939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.442964 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.444535 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.448085 (+ 3550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.448088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.448090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.448185 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.448521 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.448527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.448531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.448740 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.448848 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.449266 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.449376 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 00:46:21.450567 17964 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.450721 17964 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:46:21.451335 17985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.451483 17985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 00:46:21.453820 18001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:46:21.453889 18001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:46:21.455852 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:21.438920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.29.84.188:33291)
0806 00:46:21.439007 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:21.439011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:21.446672 (+ 7661us) server_negotiation.cc:407] Connection header received
0806 00:46:21.446832 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:21.446834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:21.446873 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:21.446917 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:21.447752 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.448455 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.449885 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:21.450243 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:21.450314 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:21.452179 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:21.452184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:21.452188 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:21.452211 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:21.453876 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.454420 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.454442 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.454444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.454544 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:21.454866 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:21.454869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:21.454871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:21.455087 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:21.455197 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:21.455579 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 00:46:21.455682 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 00:46:21.457052 17586 master_runner.cc:263] No leader master found from master 74f9b3805c1e480682f1e4034e2702d7
I20260806 00:46:21.457115 17586 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:46:21.586246 17964 raft_consensus.cc:482] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:21.586416 17964 raft_consensus.cc:504] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } }
I20260806 00:46:21.587395 17964 leader_election.cc:288] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74f9b3805c1e480682f1e4034e2702d7 (127.29.84.190:42561), a5305f8aa0004ed3a1cd91dbda3448a4 (127.29.84.189:33519)
I20260806 00:46:21.588433 17644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f69a71371b2e49e79814aad2847a2222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74f9b3805c1e480682f1e4034e2702d7" is_pre_election: true
I20260806 00:46:21.588759 17644 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69a71371b2e49e79814aad2847a2222 in term 0.
I20260806 00:46:21.589138 17761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f69a71371b2e49e79814aad2847a2222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" is_pre_election: true
I20260806 00:46:21.589378 17761 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69a71371b2e49e79814aad2847a2222 in term 0.
I20260806 00:46:21.590039 17835 leader_election.cc:302] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74f9b3805c1e480682f1e4034e2702d7, f69a71371b2e49e79814aad2847a2222; no voters:
I20260806 00:46:21.590598 17964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:46:21.590685 17964 raft_consensus.cc:482] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:46:21.590737 17964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:21.592206 17964 raft_consensus.cc:504] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } }
I20260806 00:46:21.592552 17964 leader_election.cc:288] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [CANDIDATE]: Term 1 election: Requested vote from peers 74f9b3805c1e480682f1e4034e2702d7 (127.29.84.190:42561), a5305f8aa0004ed3a1cd91dbda3448a4 (127.29.84.189:33519)
I20260806 00:46:21.593335 17761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f69a71371b2e49e79814aad2847a2222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4"
I20260806 00:46:21.593478 17761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:21.594667 17761 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69a71371b2e49e79814aad2847a2222 in term 1.
I20260806 00:46:21.595027 17644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f69a71371b2e49e79814aad2847a2222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74f9b3805c1e480682f1e4034e2702d7"
I20260806 00:46:21.595172 17644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:21.596349 17644 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69a71371b2e49e79814aad2847a2222 in term 1.
I20260806 00:46:21.596719 17835 leader_election.cc:302] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74f9b3805c1e480682f1e4034e2702d7, f69a71371b2e49e79814aad2847a2222; no voters:
I20260806 00:46:21.596987 17964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:46:21.597218 17964 raft_consensus.cc:686] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [term 1 LEADER]: Becoming Leader. State: Replica: f69a71371b2e49e79814aad2847a2222, State: Running, Role: LEADER
I20260806 00:46:21.597574 17964 consensus_queue.cc:237] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } }
I20260806 00:46:21.599277 17964 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f69a71371b2e49e79814aad2847a2222. Latest consensus state: current_term: 1 leader_uuid: "f69a71371b2e49e79814aad2847a2222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.599435 17964 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:21.603634 18008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:46:21.604553 18008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:46:21.609367 17644 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [term 1 FOLLOWER]: Refusing update from remote peer f69a71371b2e49e79814aad2847a2222: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:21.609398 17761 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [term 1 FOLLOWER]: Refusing update from remote peer f69a71371b2e49e79814aad2847a2222: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:21.610065 18006 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:46:21.610306 17964 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:46:21.613921 17930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f69a71371b2e49e79814aad2847a2222. Latest consensus state: current_term: 1 leader_uuid: "f69a71371b2e49e79814aad2847a2222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.613921 17985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f69a71371b2e49e79814aad2847a2222. Latest consensus state: current_term: 1 leader_uuid: "f69a71371b2e49e79814aad2847a2222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.614035 17930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:46:21.614060 17985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:46:21.618618 17964 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f69a71371b2e49e79814aad2847a2222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.618717 17964 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:21.620009 17985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f69a71371b2e49e79814aad2847a2222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.620115 17985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:46:21.622272 18008 catalog_manager.cc:1347] Generated new cluster ID: 596bbc370c184b1baf1dba8b99297d92
I20260806 00:46:21.622352 18008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:46:21.628684 18013 mvcc.cc:205] Tried to move back new op lower bound from 7315362535865630720 to 7315362535826636800. Current Snapshot: MvccSnapshot[applied={T|T < 7315362535865630720 or (T in {7315362535865630720})}]
I20260806 00:46:21.632808 18006 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f69a71371b2e49e79814aad2847a2222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.632934 18006 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222 [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:21.633917 17930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f69a71371b2e49e79814aad2847a2222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74f9b3805c1e480682f1e4034e2702d7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42561 } } peers { permanent_uuid: "a5305f8aa0004ed3a1cd91dbda3448a4" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 33519 } } peers { permanent_uuid: "f69a71371b2e49e79814aad2847a2222" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 33291 } } }
I20260806 00:46:21.634027 17930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:46:21.634500 18011 mvcc.cc:205] Tried to move back new op lower bound from 7315362535865630720 to 7315362535826636800. Current Snapshot: MvccSnapshot[applied={T|T < 7315362535865630720}]
I20260806 00:46:21.644450 18008 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:46:21.646127 18008 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:46:21.652586 18008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f69a71371b2e49e79814aad2847a2222: Generated new TSK 0
I20260806 00:46:21.653376 18008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:46:22.444664 17957 catalog_manager.cc:1259] Loaded cluster ID: 596bbc370c184b1baf1dba8b99297d92
I20260806 00:46:22.444789 17957 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4: loading cluster ID for follower catalog manager: success
I20260806 00:46:22.446833 17957 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4: acquiring CA information for follower catalog manager: success
I20260806 00:46:22.448019 17957 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5305f8aa0004ed3a1cd91dbda3448a4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:46:22.454862 18001 catalog_manager.cc:1259] Loaded cluster ID: 596bbc370c184b1baf1dba8b99297d92
I20260806 00:46:22.454918 18001 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7: loading cluster ID for follower catalog manager: success
I20260806 00:46:22.456347 18001 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7: acquiring CA information for follower catalog manager: success
I20260806 00:46:22.457343 18001 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 74f9b3805c1e480682f1e4034e2702d7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:46:22.712018 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 17586
I20260806 00:46:22.726255 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 17699
I20260806 00:46:22.736140 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 17809
2026-08-06T00:46:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3299 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:46:22 dist-test-slave-n326 krb5kdc[18034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:46:22 dist-test-slave-n326 krb5kdc[18034](info): set up 2 sockets
Aug 06 00:46:22 dist-test-slave-n326 krb5kdc[18034](info): commencing operation
krb5kdc: starting...
W20260806 00:46:23.928650 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:46:24 dist-test-slave-n326 krb5kdc[18034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:46:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:46:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:46:24.046298 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38511
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:38511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:24.153774 18054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:24.154043 18054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:24.154110 18054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:24.157891 18054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:24.157967 18054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:24.158003 18054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:24.158038 18054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:24.158072 18054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:24.163026 18054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:38511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:24.163924 18054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:24.164824 18054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:24.172763 18059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:24.172747 18063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:24.172904 18054 server_base.cc:1034] running on GCE node
W20260806 00:46:24.172940 18061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:24.173563 18054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:24.174536 18054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:24.175712 18054 hybrid_clock.cc:648] HybridClock initialized: now 1785977184175689 us; error 35 us; skew 500 ppm
Aug 06 00:46:24 dist-test-slave-n326 krb5kdc[18034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977184, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:24.178305 18054 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:46:24.179225 18054 webserver.cc:460] Webserver started at http://127.29.84.190:37209/ using document root <none> and password file <none>
I20260806 00:46:24.179744 18054 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:24.179836 18054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:24.180063 18054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:24.182102 18054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "57dcd9caf3ea4510ab3f6576942bc881"
format_stamp: "Formatted at 2026-08-06 00:46:24 on dist-test-slave-n326"
server_key: "e74a7d11add622ac5c914a091bfcb503"
server_key_iv: "73831c11f3626352e97fd654c7932602"
server_key_version: "encryptionkey@0"
I20260806 00:46:24.182673 18054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "57dcd9caf3ea4510ab3f6576942bc881"
format_stamp: "Formatted at 2026-08-06 00:46:24 on dist-test-slave-n326"
server_key: "e74a7d11add622ac5c914a091bfcb503"
server_key_iv: "73831c11f3626352e97fd654c7932602"
server_key_version: "encryptionkey@0"
I20260806 00:46:24.186198 18054 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:46:24.188526 18080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:24.189637 18054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:46:24.189786 18054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "57dcd9caf3ea4510ab3f6576942bc881"
format_stamp: "Formatted at 2026-08-06 00:46:24 on dist-test-slave-n326"
server_key: "e74a7d11add622ac5c914a091bfcb503"
server_key_iv: "73831c11f3626352e97fd654c7932602"
server_key_version: "encryptionkey@0"
I20260806 00:46:24.189898 18054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:24.206851 18054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:24.210013 18054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:24.210207 18054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:24.217063 18054 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:38511
I20260806 00:46:24.217084 18161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:38511 every 8 connection(s)
I20260806 00:46:24.217942 18054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:46:24.220516 18162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:24.221501 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 18054
I20260806 00:46:24.221627 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:46:24.221961 30034 external_mini_cluster.cc:1428] Setting key cd60573b87fc088676bb602331d69f29
I20260806 00:46:24.226557 18162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881: Bootstrap starting.
I20260806 00:46:24.228718 18162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:24.229434 18162 log.cc:824] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:46:24 dist-test-slave-n326 krb5kdc[18034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:24.231487 18162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881: No bootstrap required, opened a new log
I20260806 00:46:24.236034 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:24.223283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33538 (local address 127.29.84.190:38511)
0806 00:46:24.223719 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:24.223733 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:24.223765 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:46:24.224363 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:24.224378 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:24.224683 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:24.225004 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:24.225791 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:24.226819 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:24.227764 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:24.228207 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:24.228377 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:24.231021 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:24.231027 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:24.231038 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:24.231102 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:24.233080 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:24.233494 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:24.233500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:24.233507 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:24.233670 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:24.233934 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:24.233937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:24.233939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:24.234406 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:24.234588 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:24.234847 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 00:46:24.235146 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":132,"threads_started":1}
I20260806 00:46:24.236073 18162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57dcd9caf3ea4510ab3f6576942bc881" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38511 } }
I20260806 00:46:24.236311 18162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:24.236378 18162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57dcd9caf3ea4510ab3f6576942bc881, State: Initialized, Role: FOLLOWER
I20260806 00:46:24.236817 18162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57dcd9caf3ea4510ab3f6576942bc881" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38511 } }
I20260806 00:46:24.236986 18162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:46:24.237057 18162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:46:24.237152 18162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:24.238137 18162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57dcd9caf3ea4510ab3f6576942bc881" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38511 } }
I20260806 00:46:24.238480 18162 leader_election.cc:302] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57dcd9caf3ea4510ab3f6576942bc881; no voters:
I20260806 00:46:24.238790 18162 leader_election.cc:288] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:46:24.238827 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:46:24.238855 18171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:46:24.239164 18171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [term 1 LEADER]: Becoming Leader. State: Replica: 57dcd9caf3ea4510ab3f6576942bc881, State: Running, Role: LEADER
I20260806 00:46:24.239455 18171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57dcd9caf3ea4510ab3f6576942bc881" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38511 } }
I20260806 00:46:24.240104 18162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:24.240634 18171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57dcd9caf3ea4510ab3f6576942bc881. Latest consensus state: current_term: 1 leader_uuid: "57dcd9caf3ea4510ab3f6576942bc881" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57dcd9caf3ea4510ab3f6576942bc881" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38511 } } }
I20260806 00:46:24.240759 18171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:24.241149 18173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57dcd9caf3ea4510ab3f6576942bc881" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57dcd9caf3ea4510ab3f6576942bc881" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38511 } } }
I20260806 00:46:24.241267 18173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881 [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:24.242437 18178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:46:24.244820 18178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:46:24.249686 18178 catalog_manager.cc:1347] Generated new cluster ID: 4818c6d9b9d94c1abb0473c7223de951
I20260806 00:46:24.249747 18178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:46:24.257637 18178 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:46:24.258477 18178 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:46:24.267176 18178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57dcd9caf3ea4510ab3f6576942bc881: Generated new TSK 0
I20260806 00:46:24.267755 18178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:46:24.295288 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 18054
2026-08-06T00:46:24Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1563 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:46:24 dist-test-slave-n326 krb5kdc[18201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:46:24 dist-test-slave-n326 krb5kdc[18201](info): set up 2 sockets
Aug 06 00:46:24 dist-test-slave-n326 krb5kdc[18201](info): commencing operation
krb5kdc: starting...
W20260806 00:46:25.495119 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:46:25 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:46:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:46:25.615736 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40379
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:40379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:25.720670 18219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:25.720914 18219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:25.720983 18219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:25.724879 18219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:25.724958 18219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:25.724993 18219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:25.725028 18219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:25.725059 18219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:25.730182 18219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:40379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:25.731099 18219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:25.732018 18219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:25.740294 18225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:25.740303 18226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:25.740396 18230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:25.740442 18219 server_base.cc:1034] running on GCE node
I20260806 00:46:25.741030 18219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:25.741982 18219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:25.743193 18219 hybrid_clock.cc:648] HybridClock initialized: now 1785977185743170 us; error 29 us; skew 500 ppm
Aug 06 00:46:25 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977185, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:25.745744 18219 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:46:25.746502 18219 webserver.cc:460] Webserver started at http://127.29.84.190:37995/ using document root <none> and password file <none>
I20260806 00:46:25.746965 18219 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:25.747023 18219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:25.747246 18219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:25.749039 18219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "20bd538cd4044290b0dc4d9421d31fbd"
format_stamp: "Formatted at 2026-08-06 00:46:25 on dist-test-slave-n326"
server_key: "e167bb4504100f4f00ddd22b10137d0a"
server_key_iv: "4fbb240b0a9250f76072bd1975b24889"
server_key_version: "encryptionkey@0"
I20260806 00:46:25.749552 18219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "20bd538cd4044290b0dc4d9421d31fbd"
format_stamp: "Formatted at 2026-08-06 00:46:25 on dist-test-slave-n326"
server_key: "e167bb4504100f4f00ddd22b10137d0a"
server_key_iv: "4fbb240b0a9250f76072bd1975b24889"
server_key_version: "encryptionkey@0"
I20260806 00:46:25.753108 18219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:46:25.755205 18240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:25.756305 18219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:25.756449 18219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "20bd538cd4044290b0dc4d9421d31fbd"
format_stamp: "Formatted at 2026-08-06 00:46:25 on dist-test-slave-n326"
server_key: "e167bb4504100f4f00ddd22b10137d0a"
server_key_iv: "4fbb240b0a9250f76072bd1975b24889"
server_key_version: "encryptionkey@0"
I20260806 00:46:25.756554 18219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:25.786654 18219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:25.789201 18219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:25.789410 18219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:25.796027 18219 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:40379
I20260806 00:46:25.796316 18324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:40379 every 8 connection(s)
I20260806 00:46:25.796936 18219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:46:25.799566 18328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:25.801229 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 18219
I20260806 00:46:25.801383 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:46:25.801690 30034 external_mini_cluster.cc:1428] Setting key cb4d916f2e3a25652af7f8013a395720
I20260806 00:46:25.805600 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Bootstrap starting.
I20260806 00:46:25.807787 18328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:25.808570 18328 log.cc:824] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:46:25 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:25.810549 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: No bootstrap required, opened a new log
I20260806 00:46:25.814014 18328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:25.814266 18328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:25.814399 18328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20bd538cd4044290b0dc4d9421d31fbd, State: Initialized, Role: FOLLOWER
I20260806 00:46:25.814867 18328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:25.815021 18328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:46:25.815138 18328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:46:25.815250 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:25.802999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45352 (local address 127.29.84.190:40379)
0806 00:46:25.803424 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:25.803438 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:25.803472 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:46:25.804141 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:25.804156 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:25.804449 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:25.804766 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:25.805555 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:25.806543 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:25.807479 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:25.807980 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:25.808159 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:25.810734 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:25.810739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:25.810751 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:25.810776 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:25.812408 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:25.812860 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:25.812865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:25.812869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:25.812972 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:25.813259 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:25.813262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:25.813264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:25.813620 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:25.813830 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:25.814067 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 00:46:25.814386 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":121,"threads_started":1}
I20260806 00:46:25.815264 18328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:25.816383 18328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:25.816730 18328 leader_election.cc:302] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20bd538cd4044290b0dc4d9421d31fbd; no voters:
I20260806 00:46:25.817063 18328 leader_election.cc:288] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:46:25.817145 18335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:46:25.817327 18335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 1 LEADER]: Becoming Leader. State: Replica: 20bd538cd4044290b0dc4d9421d31fbd, State: Running, Role: LEADER
I20260806 00:46:25.817662 18335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:25.818262 18328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:25.819324 18336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20bd538cd4044290b0dc4d9421d31fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } } }
I20260806 00:46:25.819698 18336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:25.819597 18337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20bd538cd4044290b0dc4d9421d31fbd. Latest consensus state: current_term: 1 leader_uuid: "20bd538cd4044290b0dc4d9421d31fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } } }
I20260806 00:46:25.820027 18337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:25.820170 18344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:46:25.823009 18344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:46:25.828250 18344 catalog_manager.cc:1347] Generated new cluster ID: c95f8ce2141e40a3ae5cb480400aff27
I20260806 00:46:25.828315 18344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:46:25.844087 18344 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:46:25.845156 18344 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:46:25.856870 18344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Generated new TSK 0
I20260806 00:46:25.857544 18344 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:46:25.894505 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40379
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:26.001775 18361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:26.001986 18361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:26.002027 18361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:26.005753 18361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:26.005805 18361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:26.005858 18361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:46:26.010629 18361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:26.011509 18361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:26.012365 18361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:26.022197 18375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:26.022212 18373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:26.022462 18361 server_base.cc:1034] running on GCE node
W20260806 00:46:26.022197 18369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:26.022958 18361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:26.023608 18361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:26.024792 18361 hybrid_clock.cc:648] HybridClock initialized: now 1785977186024778 us; error 25 us; skew 500 ppm
Aug 06 00:46:26 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977186, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:26.027552 18361 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:46:26.028367 18361 webserver.cc:460] Webserver started at http://127.29.84.129:46297/ using document root <none> and password file <none>
I20260806 00:46:26.028890 18361 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:26.028978 18361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:26.029201 18361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:26.031247 18361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "bdfd588904954ef7af4c31cc15c1f9d8"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "4e10e0877c1d72d2b61e44819cb50025"
server_key_iv: "1e773fec96dbae718444bd715ea198bd"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.031803 18361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "bdfd588904954ef7af4c31cc15c1f9d8"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "4e10e0877c1d72d2b61e44819cb50025"
server_key_iv: "1e773fec96dbae718444bd715ea198bd"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.035385 18361 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:46:26.037613 18383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.038754 18361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:26.038887 18361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "bdfd588904954ef7af4c31cc15c1f9d8"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "4e10e0877c1d72d2b61e44819cb50025"
server_key_iv: "1e773fec96dbae718444bd715ea198bd"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.038990 18361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:26.057765 18361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:26.060470 18361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:26.060662 18361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:26.061250 18361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:26.062013 18361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:26.062084 18361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.062152 18361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:26.062196 18361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.071043 18361 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:46747
I20260806 00:46:26.071110 18546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:46747 every 8 connection(s)
I20260806 00:46:26.071838 18361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:46:26.080201 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 18361
I20260806 00:46:26.080338 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:46:26.080608 30034 external_mini_cluster.cc:1428] Setting key 643acaad563758f89c346eabb69f2a0f
Aug 06 00:46:26 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977186, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:26.085165 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.073350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:49737 (local address 127.29.84.190:40379)
0806 00:46:26.073476 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.073479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.074210 (+ 731us) server_negotiation.cc:407] Connection header received
0806 00:46:26.075586 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.075589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.075645 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.075747 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:26.076795 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.077332 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.078492 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.078734 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.078842 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.081362 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:26.081366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:26.081368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:26.081405 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:26.082941 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:26.083625 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:26.083628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:26.083629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:26.083709 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:26.084096 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:26.084098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:26.084099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:26.084298 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:26.084403 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.084874 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.085002 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:46:26.085813 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.073674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:49737)
0806 00:46:26.074075 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:26.074116 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:26.075391 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:26.075902 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:26.075908 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:26.076214 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:26.076626 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.076638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.077470 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.077476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.078363 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.078373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.078889 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.078893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.079052 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.079529 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:26.079544 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:26.081183 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:26.083095 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:26.083103 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:26.083119 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:26.083503 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:26.083800 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:26.083806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:26.083811 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:26.083994 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:26.084408 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:26.084415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:26.084647 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 00:46:26.084818 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 00:46:26.085100 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":112,"threads_started":1}
I20260806 00:46:26.086911 18551 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40379
I20260806 00:46:26.087178 18551 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:26.087723 18551 heartbeater.cc:511] Master 127.29.84.190:40379 requested a full tablet report, sending...
I20260806 00:46:26.089125 18266 ts_manager.cc:194] Registered new tserver with Master: bdfd588904954ef7af4c31cc15c1f9d8 (127.29.84.129:46747)
I20260806 00:46:26.090194 18266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:49737
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:46:26.109737 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40379
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:26.217941 18558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:26.218150 18558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:26.218217 18558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:26.222460 18558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:26.222509 18558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:26.222611 18558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:46:26.227602 18558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:26.228384 18558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:26.229257 18558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:26.238247 18575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:26.238263 18565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:26.238247 18564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:26.238492 18558 server_base.cc:1034] running on GCE node
I20260806 00:46:26.238889 18558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:26.239516 18558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:26.240675 18558 hybrid_clock.cc:648] HybridClock initialized: now 1785977186240662 us; error 24 us; skew 500 ppm
Aug 06 00:46:26 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977186, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:26.243532 18558 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:46:26.244365 18558 webserver.cc:460] Webserver started at http://127.29.84.130:46783/ using document root <none> and password file <none>
I20260806 00:46:26.244880 18558 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:26.244940 18558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:26.245191 18558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:26.247011 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "18c0c79f06004794aa77401629406271"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "9fd4a8505912b45be5158dcbfc82e1d4"
server_key_iv: "429b882800d38d66effb39e2c20579e8"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.247586 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "18c0c79f06004794aa77401629406271"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "9fd4a8505912b45be5158dcbfc82e1d4"
server_key_iv: "429b882800d38d66effb39e2c20579e8"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.250967 18558 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:46:26.253235 18588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.254364 18558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:26.254504 18558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "18c0c79f06004794aa77401629406271"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "9fd4a8505912b45be5158dcbfc82e1d4"
server_key_iv: "429b882800d38d66effb39e2c20579e8"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.254606 18558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:26.271965 18558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:26.275236 18558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:26.275426 18558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:26.276013 18558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:26.276801 18558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:26.276871 18558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.276939 18558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:26.276980 18558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.286130 18558 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:38709
I20260806 00:46:26.286154 18741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:38709 every 8 connection(s)
I20260806 00:46:26.286907 18558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:46:26.295469 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 18558
I20260806 00:46:26.295624 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:46:26.295981 30034 external_mini_cluster.cc:1428] Setting key b5fe827a73389e71cf3fa7e1d6a8cbfe
Aug 06 00:46:26 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977186, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:26.301106 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.288606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:37109 (local address 127.29.84.190:40379)
0806 00:46:26.288757 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.288761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.289458 (+ 697us) server_negotiation.cc:407] Connection header received
0806 00:46:26.290950 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.290954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.291019 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.291162 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:26.292333 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.292898 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.294018 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.294248 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.294362 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.297316 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:26.297321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:26.297342 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:26.297372 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:26.299166 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:26.299641 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:26.299648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:26.299653 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:26.299756 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:26.300094 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:26.300098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:26.300100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:26.300280 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:26.300392 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.300837 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.300952 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:46:26.301826 18745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.288892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:37109)
0806 00:46:26.289305 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:26.289344 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:26.290710 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:26.291333 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:26.291340 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:26.291704 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:26.292171 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.292184 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.293026 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.293032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.293888 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.293897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.294399 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.294403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.294569 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.295214 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:26.295255 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:26.297132 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:26.299282 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:26.299286 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:26.299296 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:26.299537 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:26.299857 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:26.299859 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:26.299860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:26.299980 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:26.300391 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:26.300398 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:26.300616 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 00:46:26.300798 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 00:46:26.301070 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":108,"threads_started":1}
I20260806 00:46:26.302933 18742 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40379
I20260806 00:46:26.303254 18742 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:26.303864 18742 heartbeater.cc:511] Master 127.29.84.190:40379 requested a full tablet report, sending...
I20260806 00:46:26.305052 18266 ts_manager.cc:194] Registered new tserver with Master: 18c0c79f06004794aa77401629406271 (127.29.84.130:38709)
I20260806 00:46:26.305903 18266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:37109
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:46:26.326676 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40379
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:26.434351 18748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:26.434680 18748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:26.434746 18748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:26.438817 18748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:26.438868 18748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:26.438925 18748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:46:26.444036 18748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:26.444898 18748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:26.445775 18748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:26.454756 18755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:26.454754 18756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:26.454973 18748 server_base.cc:1034] running on GCE node
W20260806 00:46:26.454974 18759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:26.455531 18748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:26.456110 18748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:26.457278 18748 hybrid_clock.cc:648] HybridClock initialized: now 1785977186457265 us; error 24 us; skew 500 ppm
Aug 06 00:46:26 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977186, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:26.460054 18748 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:46:26.460889 18748 webserver.cc:460] Webserver started at http://127.29.84.131:36611/ using document root <none> and password file <none>
I20260806 00:46:26.461397 18748 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:26.461519 18748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:26.461709 18748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:26.463608 18748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "ff89fec6438e4c6aab562610f89a3c3c"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "d707e76dd06f9ad382dc8c1e7caa2358"
server_key_iv: "175cec83715799d3b4213d8012a13bd0"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.464110 18748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "ff89fec6438e4c6aab562610f89a3c3c"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "d707e76dd06f9ad382dc8c1e7caa2358"
server_key_iv: "175cec83715799d3b4213d8012a13bd0"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.467547 18748 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:46:26.470367 18768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.471542 18748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:46:26.471671 18748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "ff89fec6438e4c6aab562610f89a3c3c"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "d707e76dd06f9ad382dc8c1e7caa2358"
server_key_iv: "175cec83715799d3b4213d8012a13bd0"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.471777 18748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:26.492835 18748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:26.495411 18748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:26.495604 18748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:26.496140 18748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:26.496903 18748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:26.496980 18748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.497045 18748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:26.497097 18748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.505501 18748 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:46279
I20260806 00:46:26.505527 18933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:46279 every 8 connection(s)
I20260806 00:46:26.506276 18748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:46:26.512326 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 18748
I20260806 00:46:26.512450 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:46:26.512727 30034 external_mini_cluster.cc:1428] Setting key fd2dcd47fa45b0f9a8f6a63456800972
Aug 06 00:46:26 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977186, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:26.519734 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.507946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:54197 (local address 127.29.84.190:40379)
0806 00:46:26.508070 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.508074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.508848 (+ 774us) server_negotiation.cc:407] Connection header received
0806 00:46:26.510092 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.510095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.510143 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.510217 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:26.511326 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.511858 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.513387 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.513724 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.513809 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.516377 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:26.516380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:26.516382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:26.516405 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:26.517804 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:26.518311 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:26.518314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:26.518315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:26.518388 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:26.518712 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:26.518715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:26.518716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:26.518931 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:26.519027 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.519532 (+ 505us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.519624 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:46:26.520491 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.508275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:54197)
0806 00:46:26.508706 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:26.508745 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:26.509907 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:26.510361 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:26.510367 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:26.510695 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:26.511194 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.511209 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.511960 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.511963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.513264 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.513274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.513888 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.513893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.514074 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.514652 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:26.514667 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:26.516230 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:26.517907 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:26.517911 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:26.517922 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:26.518229 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:26.518481 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:26.518483 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:26.518484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:26.518612 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:26.519037 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:26.519041 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:26.519329 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 00:46:26.519481 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 00:46:26.519704 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":117,"threads_started":1}
I20260806 00:46:26.521600 18934 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40379
I20260806 00:46:26.521814 18934 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:26.522233 18934 heartbeater.cc:511] Master 127.29.84.190:40379 requested a full tablet report, sending...
I20260806 00:46:26.523190 18266 ts_manager.cc:194] Registered new tserver with Master: ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279)
I20260806 00:46:26.523698 18266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:54197
I20260806 00:46:26.526379 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:46:26.534793 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.528279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.29.84.190:40379)
0806 00:46:26.528404 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.528408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.528544 (+ 136us) server_negotiation.cc:407] Connection header received
0806 00:46:26.528631 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.528633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.528667 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.528727 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:26.529323 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.529789 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.530542 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.530778 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.530853 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.532043 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:26.532045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:26.532047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:26.532068 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:26.533404 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:26.533763 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:26.533766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:26.533767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:26.533835 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:26.534050 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:26.534052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:26.534053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:26.534226 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:26.534325 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.534507 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.534586 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:46:26.537256 18266 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:46:26.540097 18266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:46:26.553471 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.548063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:46747 (local address 127.0.0.1:54676)
0806 00:46:26.548392 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:26.548457 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:26.548576 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:26.549507 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:26.549512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:26.549535 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:26.549662 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.549672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.550897 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.550903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.551900 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.551909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.552960 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.552964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.553080 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.553123 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 00:46:26.553230 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 00:46:26.553334 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
I20260806 00:46:26.553504 18949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.548194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.29.84.129:46747)
0806 00:46:26.548985 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.549004 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.549031 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:46:26.549113 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.549118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.549326 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.549522 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:26.549803 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.550772 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.552025 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.552836 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.552942 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.553085 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.553272 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.553363 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":74,"threads_started":1}
I20260806 00:46:26.554387 18950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.548238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:38709 (local address 127.0.0.1:50706)
0806 00:46:26.549052 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:26.549068 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:26.549156 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:26.549737 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:26.549740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:26.549756 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:26.550071 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.550080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.551747 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.551751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.552864 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.552873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.554012 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.554016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.554102 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.554122 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:46:26.554199 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 00:46:26.554264 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260806 00:46:26.554447 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.548663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.29.84.130:38709)
0806 00:46:26.549276 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.549289 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.549310 (+ 21us) server_negotiation.cc:407] Connection header received
0806 00:46:26.549387 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.549392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.549578 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.550232 (+ 654us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:26.550409 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.551598 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.553035 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.553859 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.554010 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.554143 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.554224 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.554307 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":111,"threads_started":1}
I20260806 00:46:26.554973 18953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.549126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:46279 (local address 127.0.0.1:60636)
0806 00:46:26.549786 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:26.549799 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:26.549879 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:26.550399 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:26.550402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:26.550421 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:26.550515 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.550520 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.551847 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.551851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.552987 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.552997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.554565 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.554568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.554657 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.554685 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:46:26.554783 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 00:46:26.554853 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":87,"threads_started":1}
I20260806 00:46:26.555157 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.549405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60636 (local address 127.29.84.131:46279)
0806 00:46:26.549988 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.550001 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.550022 (+ 21us) server_negotiation.cc:407] Connection header received
0806 00:46:26.550085 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.550089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.550253 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.550396 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:26.550635 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.551712 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.553132 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.553930 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.554034 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.554175 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.554833 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.554920 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":102,"threads_started":1}
I20260806 00:46:26.556926 18455 tablet_service.cc:1467] Processing CreateTablet for tablet d71b91e57ab048d886bd2d1afd9f9a40 (DEFAULT_TABLE table=test-table [id=c2ca4a3dcf9f49a28c329953613b1f86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:26.557163 18650 tablet_service.cc:1467] Processing CreateTablet for tablet d71b91e57ab048d886bd2d1afd9f9a40 (DEFAULT_TABLE table=test-table [id=c2ca4a3dcf9f49a28c329953613b1f86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:26.557519 18839 tablet_service.cc:1467] Processing CreateTablet for tablet d71b91e57ab048d886bd2d1afd9f9a40 (DEFAULT_TABLE table=test-table [id=c2ca4a3dcf9f49a28c329953613b1f86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:26.558048 18650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71b91e57ab048d886bd2d1afd9f9a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:26.558048 18455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71b91e57ab048d886bd2d1afd9f9a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:26.558511 18839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71b91e57ab048d886bd2d1afd9f9a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:26.564772 18955 tablet_bootstrap.cc:492] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: Bootstrap starting.
I20260806 00:46:26.565590 18958 tablet_bootstrap.cc:492] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: Bootstrap starting.
I20260806 00:46:26.565907 18957 tablet_bootstrap.cc:492] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Bootstrap starting.
I20260806 00:46:26.567327 18955 tablet_bootstrap.cc:654] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:26.567792 18957 tablet_bootstrap.cc:654] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:26.568087 18955 log.cc:824] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:26.568241 18958 tablet_bootstrap.cc:654] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:26.569075 18957 log.cc:824] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:26.569075 18958 log.cc:824] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:26.570057 18955 tablet_bootstrap.cc:492] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: No bootstrap required, opened a new log
I20260806 00:46:26.570246 18955 ts_tablet_manager.cc:1397] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:46:26.570952 18958 tablet_bootstrap.cc:492] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: No bootstrap required, opened a new log
I20260806 00:46:26.570952 18957 tablet_bootstrap.cc:492] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: No bootstrap required, opened a new log
I20260806 00:46:26.571143 18957 ts_tablet_manager.cc:1397] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:46:26.571156 18958 ts_tablet_manager.cc:1397] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:46:26.573089 18955 raft_consensus.cc:348] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:26.573318 18955 raft_consensus.cc:374] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:26.573370 18955 raft_consensus.cc:729] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff89fec6438e4c6aab562610f89a3c3c, State: Initialized, Role: FOLLOWER
I20260806 00:46:26.573797 18955 consensus_queue.cc:260] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:26.574419 18934 heartbeater.cc:503] Master 127.29.84.190:40379 was elected leader, sending a full tablet report...
I20260806 00:46:26.574761 18955 ts_tablet_manager.cc:1428] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 00:46:26.574818 18958 raft_consensus.cc:348] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:26.575167 18958 raft_consensus.cc:374] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:26.575237 18958 raft_consensus.cc:729] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c0c79f06004794aa77401629406271, State: Initialized, Role: FOLLOWER
I20260806 00:46:26.575525 18957 raft_consensus.cc:348] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:26.575672 18958 consensus_queue.cc:260] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:26.575762 18957 raft_consensus.cc:374] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:26.575819 18957 raft_consensus.cc:729] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd588904954ef7af4c31cc15c1f9d8, State: Initialized, Role: FOLLOWER
I20260806 00:46:26.576190 18957 consensus_queue.cc:260] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:26.576628 18958 ts_tablet_manager.cc:1428] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:46:26.576941 18742 heartbeater.cc:503] Master 127.29.84.190:40379 was elected leader, sending a full tablet report...
I20260806 00:46:26.577131 18957 ts_tablet_manager.cc:1428] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:46:26.577668 18551 heartbeater.cc:503] Master 127.29.84.190:40379 was elected leader, sending a full tablet report...
I20260806 00:46:26.682868 18966 raft_consensus.cc:482] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:26.683054 18966 raft_consensus.cc:504] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:26.684135 18966 leader_election.cc:288] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18c0c79f06004794aa77401629406271 (127.29.84.130:38709), ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279)
I20260806 00:46:26.688151 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.684573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:42421 (local address 127.29.84.131:46279)
0806 00:46:26.684800 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.684806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.684955 (+ 149us) server_negotiation.cc:407] Connection header received
0806 00:46:26.685086 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.685089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.685169 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.685245 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:26.685535 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.686003 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.687181 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.687574 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.687703 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.687757 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.687950 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.688019 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260806 00:46:26.688166 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.684543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:43465 (local address 127.29.84.130:38709)
0806 00:46:26.684744 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.684749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.684843 (+ 94us) server_negotiation.cc:407] Connection header received
0806 00:46:26.684992 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.684995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.685052 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.685130 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:26.685459 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.685992 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.687180 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.687578 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.687716 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.687768 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.687992 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.688068 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 00:46:26.688166 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.684425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:46279 (local address 127.29.84.129:42421)
0806 00:46:26.684841 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:26.684856 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:26.684985 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:26.685279 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:26.685281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:26.685295 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:26.685408 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.685417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.686140 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.686143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.687009 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.687018 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.687701 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.687704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.687813 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.687843 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 00:46:26.687906 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 00:46:26.688003 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":22,"thread_start_us":135,"threads_started":1}
I20260806 00:46:26.688172 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.684391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:38709 (local address 127.29.84.129:43465)
0806 00:46:26.684739 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:26.684756 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:26.684873 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:26.685168 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:26.685173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:26.685192 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:26.685327 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.685335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.686115 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.686119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.687004 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:26.687014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.687728 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.687732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:26.687842 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.687859 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 00:46:26.687951 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:46:26.688020 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":48,"thread_start_us":135,"threads_started":1}
I20260806 00:46:26.688946 18870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71b91e57ab048d886bd2d1afd9f9a40" candidate_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff89fec6438e4c6aab562610f89a3c3c" is_pre_election: true
I20260806 00:46:26.688966 18677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71b91e57ab048d886bd2d1afd9f9a40" candidate_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c0c79f06004794aa77401629406271" is_pre_election: true
I20260806 00:46:26.689240 18870 raft_consensus.cc:2454] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdfd588904954ef7af4c31cc15c1f9d8 in term 0.
I20260806 00:46:26.689257 18677 raft_consensus.cc:2454] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdfd588904954ef7af4c31cc15c1f9d8 in term 0.
I20260806 00:46:26.689782 18386 leader_election.cc:302] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 18c0c79f06004794aa77401629406271, bdfd588904954ef7af4c31cc15c1f9d8, ff89fec6438e4c6aab562610f89a3c3c; no voters:
I20260806 00:46:26.690038 18966 raft_consensus.cc:2781] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:46:26.690136 18966 raft_consensus.cc:482] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:46:26.690198 18966 raft_consensus.cc:3037] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:26.691349 18966 raft_consensus.cc:504] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:26.691723 18966 leader_election.cc:288] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 1 election: Requested vote from peers 18c0c79f06004794aa77401629406271 (127.29.84.130:38709), ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279)
I20260806 00:46:26.692085 18870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71b91e57ab048d886bd2d1afd9f9a40" candidate_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff89fec6438e4c6aab562610f89a3c3c"
I20260806 00:46:26.692085 18677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71b91e57ab048d886bd2d1afd9f9a40" candidate_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c0c79f06004794aa77401629406271"
I20260806 00:46:26.692203 18677 raft_consensus.cc:3037] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:26.692203 18870 raft_consensus.cc:3037] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:26.693420 18677 raft_consensus.cc:2454] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdfd588904954ef7af4c31cc15c1f9d8 in term 1.
I20260806 00:46:26.693420 18870 raft_consensus.cc:2454] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdfd588904954ef7af4c31cc15c1f9d8 in term 1.
I20260806 00:46:26.693709 18387 leader_election.cc:302] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c0c79f06004794aa77401629406271, bdfd588904954ef7af4c31cc15c1f9d8; no voters:
I20260806 00:46:26.693907 18966 raft_consensus.cc:2781] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:46:26.694160 18966 raft_consensus.cc:686] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 LEADER]: Becoming Leader. State: Replica: bdfd588904954ef7af4c31cc15c1f9d8, State: Running, Role: LEADER
I20260806 00:46:26.694464 18966 consensus_queue.cc:237] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:26.697722 18264 catalog_manager.cc:5495] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdfd588904954ef7af4c31cc15c1f9d8 (127.29.84.129). New cstate: current_term: 1 leader_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } health_report { overall_health: UNKNOWN } } }
I20260806 00:46:26.739900 18949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.736500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.29.84.129:46747)
0806 00:46:26.736627 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:26.736630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:26.736644 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:46:26.736787 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:26.736789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:26.736842 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:26.736928 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:46:26.737129 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.737715 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.738520 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:26.738714 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:26.738787 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:26.739030 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:46:26.739523 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:46:26.739616 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:26.739692 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 00:46:26.739736 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:46:26.745435 18677 raft_consensus.cc:1261] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [term 1 FOLLOWER]: Refusing update from remote peer bdfd588904954ef7af4c31cc15c1f9d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:26.745445 18870 raft_consensus.cc:1261] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [term 1 FOLLOWER]: Refusing update from remote peer bdfd588904954ef7af4c31cc15c1f9d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:26.746162 18969 consensus_queue.cc:1035] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:46:26.746342 18966 consensus_queue.cc:1035] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:46:26.753485 18976 mvcc.cc:205] Tried to move back new op lower bound from 7315362556902772736 to 7315362556703272960. Current Snapshot: MvccSnapshot[applied={T|T < 7315362556902772736}]
I20260806 00:46:26.760931 18263 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:46:26.761091 18263 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:46:26.763449 18263 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Sending DeleteTablet for 3 replicas of tablet d71b91e57ab048d886bd2d1afd9f9a40
I20260806 00:46:26.764137 18455 tablet_service.cc:1514] Processing DeleteTablet for tablet d71b91e57ab048d886bd2d1afd9f9a40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:26 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:54676
I20260806 00:46:26.764182 18650 tablet_service.cc:1514] Processing DeleteTablet for tablet d71b91e57ab048d886bd2d1afd9f9a40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:26 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:50706
I20260806 00:46:26.764611 18988 tablet_replica.cc:330] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: stopping tablet replica
I20260806 00:46:26.764707 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 18361
I20260806 00:46:26.764952 18988 raft_consensus.cc:2229] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:46:26.767114 18987 tablet_replica.cc:330] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: stopping tablet replica
I20260806 00:46:26.767387 18987 raft_consensus.cc:2229] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:46:26.767656 18987 raft_consensus.cc:2258] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:46:26.768649 18839 tablet_service.cc:1514] Processing DeleteTablet for tablet d71b91e57ab048d886bd2d1afd9f9a40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:26 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:60636
I20260806 00:46:26.770752 18987 ts_tablet_manager.cc:1905] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:46:26.774384 18987 ts_tablet_manager.cc:1918] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:46:26.774582 18987 log.cc:1196] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/d71b91e57ab048d886bd2d1afd9f9a40
I20260806 00:46:26.774996 18987 ts_tablet_manager.cc:1939] T d71b91e57ab048d886bd2d1afd9f9a40 P 18c0c79f06004794aa77401629406271: Deleting consensus metadata
I20260806 00:46:26.775934 18989 tablet_replica.cc:330] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: stopping tablet replica
I20260806 00:46:26.776144 18245 catalog_manager.cc:4841] TS 18c0c79f06004794aa77401629406271 (127.29.84.130:38709): tablet d71b91e57ab048d886bd2d1afd9f9a40 (table test-table [id=c2ca4a3dcf9f49a28c329953613b1f86]) successfully deleted
I20260806 00:46:26.776222 18989 raft_consensus.cc:2229] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:46:26.776489 18989 raft_consensus.cc:2258] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:46:26.777619 18989 ts_tablet_manager.cc:1905] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:46:26.779799 18989 ts_tablet_manager.cc:1918] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:46:26.779912 18989 log.cc:1196] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/d71b91e57ab048d886bd2d1afd9f9a40
I20260806 00:46:26.780216 18989 ts_tablet_manager.cc:1939] T d71b91e57ab048d886bd2d1afd9f9a40 P ff89fec6438e4c6aab562610f89a3c3c: Deleting consensus metadata
I20260806 00:46:26.781184 18242 catalog_manager.cc:4841] TS ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279): tablet d71b91e57ab048d886bd2d1afd9f9a40 (table test-table [id=c2ca4a3dcf9f49a28c329953613b1f86]) successfully deleted
W20260806 00:46:26.781709 18246 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:46747 (error 108)
I20260806 00:46:26.782467 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 18558
I20260806 00:46:26.782722 18953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:26.782394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:46747 (local address 127.0.0.1:54690)
0806 00:46:26.782533 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:26.782626 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:46747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 00:46:26.787636 18246 catalog_manager.cc:4584] TS bdfd588904954ef7af4c31cc15c1f9d8 (127.29.84.129:46747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d71b91e57ab048d886bd2d1afd9f9a40: Network error: Client connection negotiation failed: client connection to 127.29.84.129:46747: connect: Connection refused (error 111)
I20260806 00:46:26.793882 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 18748
I20260806 00:46:26.806173 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 18219
I20260806 00:46:26.817451 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40379
--webserver_interface=127.29.84.190
--webserver_port=37995
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:40379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:26.923491 18994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:26.923702 18994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:26.923738 18994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:26.927667 18994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:26.927713 18994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:26.927731 18994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:26.927747 18994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:26.927762 18994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:26.932817 18994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:40379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=37995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:26.933696 18994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:26.934598 18994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:26.947588 19001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:26.947661 19005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:26.947778 18994 server_base.cc:1034] running on GCE node
W20260806 00:46:26.947615 19003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:26.948491 18994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:26.949513 18994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:26.950724 18994 hybrid_clock.cc:648] HybridClock initialized: now 1785977186950715 us; error 29 us; skew 500 ppm
Aug 06 00:46:26 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977186, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:26.953490 18994 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:46:26.954272 18994 webserver.cc:460] Webserver started at http://127.29.84.190:37995/ using document root <none> and password file <none>
I20260806 00:46:26.954757 18994 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:26.954821 18994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:26.958256 18994 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:26.960232 19017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:26.961344 18994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:46:26.961477 18994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "20bd538cd4044290b0dc4d9421d31fbd"
format_stamp: "Formatted at 2026-08-06 00:46:25 on dist-test-slave-n326"
server_key: "e167bb4504100f4f00ddd22b10137d0a"
server_key_iv: "4fbb240b0a9250f76072bd1975b24889"
server_key_version: "encryptionkey@0"
I20260806 00:46:26.961984 18994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:26.982059 18994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:26.984675 18994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:26.984817 18994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:26.991423 18994 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:40379
I20260806 00:46:26.991685 19092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:40379 every 8 connection(s)
I20260806 00:46:26.992446 18994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:46:26.993737 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 18994
I20260806 00:46:26.994223 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:46747
--local_ip_for_outbound_sockets=127.29.84.129
--tserver_master_addrs=127.29.84.190:40379
--webserver_port=46297
--webserver_interface=127.29.84.129
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:46:26.997345 19093 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:46:26.998911 19093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Bootstrap starting.
I20260806 00:46:27.015555 19093 log.cc:824] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:27.023852 19093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:46:27.024174 19093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Bootstrap complete.
I20260806 00:46:27.028146 19093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:27.028590 19093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20bd538cd4044290b0dc4d9421d31fbd, State: Initialized, Role: FOLLOWER
I20260806 00:46:27.029093 19093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:27.029229 19093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:46:27.029318 19093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:46:27.029407 19093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:46:27.031230 19093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:27.031575 19093 leader_election.cc:302] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20bd538cd4044290b0dc4d9421d31fbd; no voters:
I20260806 00:46:27.031934 19093 leader_election.cc:288] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:46:27.032009 19097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:46:27.032203 19097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 2 LEADER]: Becoming Leader. State: Replica: 20bd538cd4044290b0dc4d9421d31fbd, State: Running, Role: LEADER
I20260806 00:46:27.032472 19097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:27.033083 19093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:27.033339 19097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20bd538cd4044290b0dc4d9421d31fbd. Latest consensus state: current_term: 2 leader_uuid: "20bd538cd4044290b0dc4d9421d31fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } } }
I20260806 00:46:27.033454 19097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:27.033787 19098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "20bd538cd4044290b0dc4d9421d31fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } } }
I20260806 00:46:27.034003 19098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:27.033795 19103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:46:27.038980 19103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:46:27.039883 19103 catalog_manager.cc:1259] Loaded cluster ID: c95f8ce2141e40a3ae5cb480400aff27
I20260806 00:46:27.039947 19103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:46:27.041280 19103 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 00:46:27.045845 19103 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 00:46:27.050863 19103 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:46:27.051589 19103 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Loaded TSK: 0
I20260806 00:46:27.052742 19103 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:46:27.108418 19095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:27.108628 19095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:27.108666 19095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:27.112588 19095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:27.112635 19095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:27.112689 19095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:46:27.117481 19095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:46747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=46297
--tserver_master_addrs=127.29.84.190:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:27.118337 19095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:27.119297 19095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:27.128212 19130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:27.128325 19129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:27.128564 19132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:27.128705 19095 server_base.cc:1034] running on GCE node
I20260806 00:46:27.129070 19095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:27.129739 19095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:27.130921 19095 hybrid_clock.cc:648] HybridClock initialized: now 1785977187130905 us; error 34 us; skew 500 ppm
Aug 06 00:46:27 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977187, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:27.139582 19095 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:46:27.140530 19095 webserver.cc:460] Webserver started at http://127.29.84.129:46297/ using document root <none> and password file <none>
I20260806 00:46:27.141057 19095 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:27.141129 19095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:27.145040 19095 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:46:27.147176 19145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:27.148397 19095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:27.148485 19095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "bdfd588904954ef7af4c31cc15c1f9d8"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "4e10e0877c1d72d2b61e44819cb50025"
server_key_iv: "1e773fec96dbae718444bd715ea198bd"
server_key_version: "encryptionkey@0"
I20260806 00:46:27.149009 19095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:27.176769 19095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:27.179601 19095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:27.179840 19095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:27.180464 19095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:27.181694 19154 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:46:27.185643 19095 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:46:27.185708 19095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260806 00:46:27.185791 19095 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:46:27.187672 19095 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:46:27.187721 19095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 00:46:27.188017 19154 tablet_bootstrap.cc:492] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Bootstrap starting.
I20260806 00:46:27.199556 19095 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:46747
I20260806 00:46:27.199656 19295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:46747 every 8 connection(s)
I20260806 00:46:27.200563 19095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:46:27.201265 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 19095
I20260806 00:46:27.201807 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:38709
--local_ip_for_outbound_sockets=127.29.84.130
--tserver_master_addrs=127.29.84.190:40379
--webserver_port=46783
--webserver_interface=127.29.84.130
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:46:27.206269 19154 log.cc:824] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:27.213675 19154 tablet_bootstrap.cc:492] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:46:27.214044 19154 tablet_bootstrap.cc:492] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Bootstrap complete.
Aug 06 00:46:27 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977187, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:27.215523 19154 ts_tablet_manager.cc:1397] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Time spent bootstrapping tablet: real 0.028s user 0.023s sys 0.004s
I20260806 00:46:27.218075 19154 raft_consensus.cc:348] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:27.218506 19154 raft_consensus.cc:729] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd588904954ef7af4c31cc15c1f9d8, State: Initialized, Role: FOLLOWER
I20260806 00:46:27.218997 19154 consensus_queue.cc:260] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } } peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } }
I20260806 00:46:27.219950 19154 ts_tablet_manager.cc:1428] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 00:46:27.221467 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.202851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:45387)
0806 00:46:27.203767 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:27.203825 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:27.205121 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:27.206582 (+ 1461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:27.206594 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:27.207202 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:27.207983 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.208010 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.209899 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.209907 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.211676 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.211693 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.212639 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.212647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.212912 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.213659 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:27.213688 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:27.215795 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:27.218493 (+ 2698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:27.218498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:27.218510 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:27.218838 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:27.219295 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:27.219299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:27.219301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:27.219513 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:27.220225 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:27.220230 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:27.220428 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 00:46:27.220568 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 00:46:27.220789 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":199,"threads_started":1}
I20260806 00:46:27.221927 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.202841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:45387 (local address 127.29.84.190:40379)
0806 00:46:27.203423 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.203445 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.204003 (+ 558us) server_negotiation.cc:407] Connection header received
0806 00:46:27.205622 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.205639 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.206036 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.206428 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:27.208201 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.209709 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.211830 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.212475 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.212633 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.216030 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:27.216040 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:27.216061 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:27.216105 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:27.218353 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.218969 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.218975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.218981 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.219170 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.219637 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.219641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.219643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.220090 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:27.220234 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.220740 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.221049 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":132,"threads_started":1}
I20260806 00:46:27.224040 19296 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40379
I20260806 00:46:27.224273 19296 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:27.224730 19296 heartbeater.cc:511] Master 127.29.84.190:40379 requested a full tablet report, sending...
I20260806 00:46:27.227671 19039 ts_manager.cc:194] Registered new tserver with Master: bdfd588904954ef7af4c31cc15c1f9d8 (127.29.84.129:46747)
I20260806 00:46:27.230832 19039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:45387
I20260806 00:46:27.233289 19296 heartbeater.cc:503] Master 127.29.84.190:40379 was elected leader, sending a full tablet report...
Aug 06 00:46:27 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977186, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:46:27.241029 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.230325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:46747 (local address 127.0.0.1:54702)
0806 00:46:27.231013 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:27.231053 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:27.231203 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:27.231711 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:27.231717 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:27.232145 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:27.232260 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.232268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.233769 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.233773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.234451 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.234461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.235133 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.235136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.235246 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.235776 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:27.235788 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:27.237233 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:27.239322 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:27.239331 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:27.239337 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:27.239566 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:27.239873 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:27.239876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:27.239878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:27.239963 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:27.240413 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:27.240428 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:27.240544 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 00:46:27.240680 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 00:46:27.240861 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":135,"threads_started":1}
I20260806 00:46:27.241083 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.230383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.29.84.129:46747)
0806 00:46:27.230755 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.230766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.231176 (+ 410us) server_negotiation.cc:407] Connection header received
0806 00:46:27.231350 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.231356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.231566 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.231703 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:27.232413 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.233628 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.234576 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.234996 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.235138 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.237368 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:27.237378 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:27.237389 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:27.237420 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:27.239210 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.239682 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.239688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.239689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.239777 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.240056 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.240061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.240062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.240295 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:27.240436 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.240770 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.240934 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":295,"thread_start_us":100,"threads_started":1}
I20260806 00:46:27.242023 19204 tablet_service.cc:1514] Processing DeleteTablet for tablet d71b91e57ab048d886bd2d1afd9f9a40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:26 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:54702
I20260806 00:46:27.243000 19312 tablet_replica.cc:330] stopping tablet replica
I20260806 00:46:27.243294 19312 raft_consensus.cc:2229] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:46:27.243527 19312 raft_consensus.cc:2258] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:46:27.244413 19312 ts_tablet_manager.cc:1905] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:46:27.247710 19312 ts_tablet_manager.cc:1918] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:46:27.247822 19312 log.cc:1196] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/d71b91e57ab048d886bd2d1afd9f9a40
I20260806 00:46:27.248092 19312 ts_tablet_manager.cc:1939] T d71b91e57ab048d886bd2d1afd9f9a40 P bdfd588904954ef7af4c31cc15c1f9d8: Deleting consensus metadata
I20260806 00:46:27.249621 19023 catalog_manager.cc:4841] TS bdfd588904954ef7af4c31cc15c1f9d8 (127.29.84.129:46747): tablet d71b91e57ab048d886bd2d1afd9f9a40 (table test-table [id=c2ca4a3dcf9f49a28c329953613b1f86]) successfully deleted
W20260806 00:46:27.325289 19301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:27.325492 19301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:27.325531 19301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:27.329417 19301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:27.329464 19301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:27.329517 19301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:46:27.334465 19301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:38709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=46783
--tserver_master_addrs=127.29.84.190:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:27.335304 19301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:27.336122 19301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:27.344995 19324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:27.345026 19320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:27.344969 19322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:27.345227 19301 server_base.cc:1034] running on GCE node
I20260806 00:46:27.345654 19301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:27.346235 19301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:27.347438 19301 hybrid_clock.cc:648] HybridClock initialized: now 1785977187347421 us; error 29 us; skew 500 ppm
Aug 06 00:46:27 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977187, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:27.350081 19301 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:46:27.350916 19301 webserver.cc:460] Webserver started at http://127.29.84.130:46783/ using document root <none> and password file <none>
I20260806 00:46:27.351517 19301 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:27.351646 19301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:27.355266 19301 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 00:46:27.357163 19334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:27.358287 19301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:27.358439 19301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "18c0c79f06004794aa77401629406271"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "9fd4a8505912b45be5158dcbfc82e1d4"
server_key_iv: "429b882800d38d66effb39e2c20579e8"
server_key_version: "encryptionkey@0"
I20260806 00:46:27.358923 19301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:27.380944 19301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:27.383530 19301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:27.383715 19301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:27.384236 19301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:27.384981 19301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:27.385053 19301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:27.385123 19301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:27.385165 19301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:27.394142 19301 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:38709
I20260806 00:46:27.394182 19490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:38709 every 8 connection(s)
I20260806 00:46:27.395030 19301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:46:27.400322 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 19301
I20260806 00:46:27.400771 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:46279
--local_ip_for_outbound_sockets=127.29.84.131
--tserver_master_addrs=127.29.84.190:40379
--webserver_port=36611
--webserver_interface=127.29.84.131
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:46:27 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977187, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:27.409363 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.396720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:57625 (local address 127.29.84.190:40379)
0806 00:46:27.396892 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.396897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.397668 (+ 771us) server_negotiation.cc:407] Connection header received
0806 00:46:27.398907 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.398910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.398956 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.399030 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:27.400148 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.400958 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.402101 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.402494 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.402625 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.405539 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:27.405542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:27.405544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:27.405568 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:27.407060 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.407803 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.407806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.407807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.407883 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.408296 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.408298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.408299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.408455 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:27.408527 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.409152 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.409247 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:46:27.410547 19493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.397072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:57625)
0806 00:46:27.397502 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:27.397564 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:27.398711 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:27.399207 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:27.399213 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:27.399554 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:27.400006 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.400018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.401096 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.401099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.401974 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.401983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.402682 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.402687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.402908 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.403526 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:27.403548 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:27.405348 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:27.407266 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:27.407275 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:27.407288 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:27.407674 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:27.407980 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:27.407985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:27.407988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:27.408187 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:27.408561 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:27.408570 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:27.408883 (+ 313us) client_negotiation.cc:769] Sending connection context
0806 00:46:27.409120 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 00:46:27.409474 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
I20260806 00:46:27.411788 19491 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40379
I20260806 00:46:27.412005 19491 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:27.412436 19491 heartbeater.cc:511] Master 127.29.84.190:40379 requested a full tablet report, sending...
I20260806 00:46:27.413442 19039 ts_manager.cc:194] Registered new tserver with Master: 18c0c79f06004794aa77401629406271 (127.29.84.130:38709)
I20260806 00:46:27.413966 19039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:57625
W20260806 00:46:27.511319 19494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:27.511548 19494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:27.511631 19494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:27.515296 19494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:27.515363 19494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:27.515444 19494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:46:27.520460 19494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:46279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=36611
--tserver_master_addrs=127.29.84.190:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:27.521349 19494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:27.522282 19494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:27.531785 19501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:27.531809 19504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:27.531793 19499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:27.532060 19494 server_base.cc:1034] running on GCE node
I20260806 00:46:27.532444 19494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:27.533111 19494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:27.534332 19494 hybrid_clock.cc:648] HybridClock initialized: now 1785977187534311 us; error 39 us; skew 500 ppm
Aug 06 00:46:27 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977187, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:27.537021 19494 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:46:27.537796 19494 webserver.cc:460] Webserver started at http://127.29.84.131:36611/ using document root <none> and password file <none>
I20260806 00:46:27.538327 19494 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:27.538439 19494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:27.542102 19494 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 00:46:27.544116 19515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:27.545241 19494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:27.545368 19494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "ff89fec6438e4c6aab562610f89a3c3c"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "d707e76dd06f9ad382dc8c1e7caa2358"
server_key_iv: "175cec83715799d3b4213d8012a13bd0"
server_key_version: "encryptionkey@0"
I20260806 00:46:27.545845 19494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:27.563637 19494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:27.566339 19494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:27.566519 19494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:27.567036 19494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:27.567838 19494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:27.567909 19494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:27.567975 19494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:27.568027 19494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:27.576356 19494 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:46279
I20260806 00:46:27.576391 19678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:46279 every 8 connection(s)
I20260806 00:46:27.577234 19494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:46:27.578567 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 19494
Aug 06 00:46:27 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977187, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:27.588557 19688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.579816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.29.84.190:40379)
0806 00:46:27.580071 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.580077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.580095 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:46:27.580136 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.580138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.580179 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.580287 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:27.581053 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.581600 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.582481 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.582724 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.582833 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.584267 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:27.584271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:27.584274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:27.584301 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:27.586026 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.586519 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.586522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.586525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.586677 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.586999 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.587004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.587031 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.587406 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:27.587677 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.588225 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.588354 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":125,"threads_started":1}
I20260806 00:46:27.591181 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.578984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:60425 (local address 127.29.84.190:40379)
0806 00:46:27.579131 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.579136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.580060 (+ 924us) server_negotiation.cc:407] Connection header received
0806 00:46:27.581455 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.581458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.581511 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.581586 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:27.582624 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.583191 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.584284 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.584508 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.584617 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.587190 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:27.587193 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:27.587196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:27.587247 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:27.589017 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.589615 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.589621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.589624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.589718 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:27.590125 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:27.590127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:27.590129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:27.590278 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:27.590354 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.590930 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.591013 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:46:27.591888 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.579427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:60425)
0806 00:46:27.579922 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:27.579971 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:27.581234 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:27.581742 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:27.581748 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:27.582052 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:27.582469 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.582480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.583317 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.583325 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.584154 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.584164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.584659 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.584664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.584815 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.585248 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:27.585262 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:27.587037 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:27.589159 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:27.589165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:27.589177 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:27.589513 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:27.589840 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:27.589843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:27.589844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:27.590025 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:27.590384 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:27.590390 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:27.590673 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 00:46:27.590860 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 00:46:27.591128 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":169,"threads_started":1}
I20260806 00:46:27.592978 19679 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40379
I20260806 00:46:27.593209 19679 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:27.593641 19679 heartbeater.cc:511] Master 127.29.84.190:40379 requested a full tablet report, sending...
I20260806 00:46:27.594537 19039 ts_manager.cc:194] Registered new tserver with Master: ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279)
I20260806 00:46:27.595049 19039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:60425
I20260806 00:46:27.602257 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:46:27.606021 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.602836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.29.84.190:40379)
0806 00:46:27.602976 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.602979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.603052 (+ 73us) server_negotiation.cc:407] Connection header received
0806 00:46:27.603248 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.603250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.603328 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.603413 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:46:27.603596 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.604038 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.604844 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.605046 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.605131 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.605397 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:46:27.605694 (+ 297us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:46:27.605780 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.605831 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.605867 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:46:27.606540 19039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:46:27.609609 19039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:46:27.616945 19204 tablet_service.cc:1467] Processing CreateTablet for tablet 629ef869d8f849feb7b926b86b28ad0d (DEFAULT_TABLE table=test-table [id=3c86150bec2b4293ac74954c8237bdc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:27.617560 19204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 629ef869d8f849feb7b926b86b28ad0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:27.620150 19154 tablet_bootstrap.cc:492] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Bootstrap starting.
I20260806 00:46:27.621028 19154 tablet_bootstrap.cc:654] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:27.621629 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.616145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:46279 (local address 127.0.0.1:60646)
0806 00:46:27.616487 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:27.616501 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:27.616590 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:27.617303 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:27.617306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:27.617323 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:27.617404 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.617411 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.618584 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.618590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.619964 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.619975 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.621294 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.621297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.621386 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.621419 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 00:46:27.621473 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 00:46:27.621522 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":24,"thread_start_us":82,"threads_started":1}
I20260806 00:46:27.621790 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.616509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.29.84.131:46279)
0806 00:46:27.616903 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.616915 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.616934 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:46:27.616994 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.616999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.617169 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.617320 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:27.617527 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.618457 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.620345 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.621165 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.621291 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.621484 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.621541 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.621629 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":109,"threads_started":1}
I20260806 00:46:27.622654 19154 tablet_bootstrap.cc:492] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: No bootstrap required, opened a new log
I20260806 00:46:27.622745 19154 ts_tablet_manager.cc:1397] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:46:27.623211 19154 raft_consensus.cc:348] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:27.623337 19154 raft_consensus.cc:374] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:27.623416 19154 raft_consensus.cc:729] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd588904954ef7af4c31cc15c1f9d8, State: Initialized, Role: FOLLOWER
I20260806 00:46:27.623577 19154 consensus_queue.cc:260] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:27.623987 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.615891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:38709 (local address 127.0.0.1:50718)
0806 00:46:27.616004 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:27.616020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:27.616096 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:27.616904 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:27.616907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:27.616928 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:27.617012 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.617018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.618747 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.618750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.619968 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.619975 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.623589 (+ 3614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.623595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.623707 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.623730 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:46:27.623782 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 00:46:27.623874 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":53}
I20260806 00:46:27.623991 19154 ts_tablet_manager.cc:1428] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:46:27.624729 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.616067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.29.84.130:38709)
0806 00:46:27.616431 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.616443 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.616465 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:46:27.616537 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.616542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.616736 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.616904 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:27.617111 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.618616 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.622555 (+ 3939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.623436 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.624275 (+ 839us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.624441 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.624486 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.624589 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":140,"threads_started":1}
I20260806 00:46:27.624759 19584 tablet_service.cc:1467] Processing CreateTablet for tablet 629ef869d8f849feb7b926b86b28ad0d (DEFAULT_TABLE table=test-table [id=3c86150bec2b4293ac74954c8237bdc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:27.625900 19584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 629ef869d8f849feb7b926b86b28ad0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:27.627517 19398 tablet_service.cc:1467] Processing CreateTablet for tablet 629ef869d8f849feb7b926b86b28ad0d (DEFAULT_TABLE table=test-table [id=3c86150bec2b4293ac74954c8237bdc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:27.628477 19398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 629ef869d8f849feb7b926b86b28ad0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:27.633986 19701 tablet_bootstrap.cc:492] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: Bootstrap starting.
I20260806 00:46:27.636096 19703 tablet_bootstrap.cc:492] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: Bootstrap starting.
I20260806 00:46:27.636827 19701 tablet_bootstrap.cc:654] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:27.637779 19701 log.cc:824] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:27.638860 19703 tablet_bootstrap.cc:654] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:27.639745 19703 log.cc:824] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:27.639964 19701 tablet_bootstrap.cc:492] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: No bootstrap required, opened a new log
I20260806 00:46:27.640166 19701 ts_tablet_manager.cc:1397] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:46:27.641453 19703 tablet_bootstrap.cc:492] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: No bootstrap required, opened a new log
I20260806 00:46:27.641649 19703 ts_tablet_manager.cc:1397] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:46:27.643195 19701 raft_consensus.cc:348] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:27.643419 19701 raft_consensus.cc:374] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:27.643496 19701 raft_consensus.cc:729] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff89fec6438e4c6aab562610f89a3c3c, State: Initialized, Role: FOLLOWER
I20260806 00:46:27.643939 19701 consensus_queue.cc:260] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:27.644574 19679 heartbeater.cc:503] Master 127.29.84.190:40379 was elected leader, sending a full tablet report...
I20260806 00:46:27.644582 19703 raft_consensus.cc:348] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:27.644822 19703 raft_consensus.cc:374] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:27.644917 19703 raft_consensus.cc:729] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c0c79f06004794aa77401629406271, State: Initialized, Role: FOLLOWER
I20260806 00:46:27.644924 19701 ts_tablet_manager.cc:1428] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:46:27.645351 19703 consensus_queue.cc:260] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:27.646203 19703 ts_tablet_manager.cc:1428] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:46:27.646327 19491 heartbeater.cc:503] Master 127.29.84.190:40379 was elected leader, sending a full tablet report...
W20260806 00:46:27.702216 19297 tablet.cc:2369] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:46:27.828131 19681 tablet.cc:2369] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:46:27.872507 19708 raft_consensus.cc:482] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:27.872699 19708 raft_consensus.cc:504] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:27.873827 19708 leader_election.cc:288] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279), bdfd588904954ef7af4c31cc15c1f9d8 (127.29.84.129:46747)
I20260806 00:46:27.877590 19493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.874008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:46279 (local address 127.29.84.130:54913)
0806 00:46:27.874202 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:27.874217 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:27.874306 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:27.874673 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:27.874676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:27.874701 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:27.874824 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.874830 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.875672 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.875676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.876494 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.876503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.877180 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.877184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.877290 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.877324 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 00:46:27.877389 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 00:46:27.877456 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 00:46:27.877647 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.874139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:54913 (local address 127.29.84.131:46279)
0806 00:46:27.874274 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.874278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.874293 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:46:27.874476 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.874480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.874554 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.874634 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:27.874962 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.875563 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.876661 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.877068 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.877156 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.877193 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.877426 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.877498 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:46:27.877775 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.874127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:46747 (local address 127.29.84.130:58829)
0806 00:46:27.874513 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:27.874529 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:27.874646 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:27.874957 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:27.874960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:27.874978 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:27.875107 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.875114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.875817 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.875819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.876664 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:27.876674 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.877352 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.877355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:27.877474 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.877494 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:46:27.877560 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 00:46:27.877635 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":68,"threads_started":1}
I20260806 00:46:27.877846 19713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.874220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:58829 (local address 127.29.84.129:46747)
0806 00:46:27.874482 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.874486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.874637 (+ 151us) server_negotiation.cc:407] Connection header received
0806 00:46:27.874741 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.874743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.874810 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.874918 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:27.875205 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.875710 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.876784 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.877245 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.877357 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.877500 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.877586 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.877689 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":60,"threads_started":1}
I20260806 00:46:27.878381 19231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "629ef869d8f849feb7b926b86b28ad0d" candidate_uuid: "18c0c79f06004794aa77401629406271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" is_pre_election: true
I20260806 00:46:27.878479 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "629ef869d8f849feb7b926b86b28ad0d" candidate_uuid: "18c0c79f06004794aa77401629406271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff89fec6438e4c6aab562610f89a3c3c" is_pre_election: true
I20260806 00:46:27.878679 19231 raft_consensus.cc:2454] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c0c79f06004794aa77401629406271 in term 0.
I20260806 00:46:27.878826 19614 raft_consensus.cc:2454] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c0c79f06004794aa77401629406271 in term 0.
I20260806 00:46:27.879171 19339 leader_election.cc:302] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c0c79f06004794aa77401629406271, bdfd588904954ef7af4c31cc15c1f9d8; no voters:
I20260806 00:46:27.879452 19708 raft_consensus.cc:2781] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:46:27.879531 19708 raft_consensus.cc:482] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:46:27.879563 19708 raft_consensus.cc:3037] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:27.880617 19708 raft_consensus.cc:504] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:27.881003 19708 leader_election.cc:288] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [CANDIDATE]: Term 1 election: Requested vote from peers ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279), bdfd588904954ef7af4c31cc15c1f9d8 (127.29.84.129:46747)
I20260806 00:46:27.881352 19231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "629ef869d8f849feb7b926b86b28ad0d" candidate_uuid: "18c0c79f06004794aa77401629406271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdfd588904954ef7af4c31cc15c1f9d8"
I20260806 00:46:27.881352 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "629ef869d8f849feb7b926b86b28ad0d" candidate_uuid: "18c0c79f06004794aa77401629406271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff89fec6438e4c6aab562610f89a3c3c"
I20260806 00:46:27.881487 19231 raft_consensus.cc:3037] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:27.881488 19614 raft_consensus.cc:3037] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:27.882762 19614 raft_consensus.cc:2454] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18c0c79f06004794aa77401629406271 in term 1.
I20260806 00:46:27.882762 19231 raft_consensus.cc:2454] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18c0c79f06004794aa77401629406271 in term 1.
I20260806 00:46:27.883044 19336 leader_election.cc:302] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c0c79f06004794aa77401629406271, ff89fec6438e4c6aab562610f89a3c3c; no voters:
I20260806 00:46:27.883283 19708 raft_consensus.cc:2781] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:46:27.883479 19708 raft_consensus.cc:686] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 1 LEADER]: Becoming Leader. State: Replica: 18c0c79f06004794aa77401629406271, State: Running, Role: LEADER
I20260806 00:46:27.883769 19708 consensus_queue.cc:237] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:27.886816 19039 catalog_manager.cc:5495] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18c0c79f06004794aa77401629406271 (127.29.84.130). New cstate: current_term: 1 leader_uuid: "18c0c79f06004794aa77401629406271" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } health_report { overall_health: UNKNOWN } } }
W20260806 00:46:27.896338 19492 tablet.cc:2369] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:46:27.912132 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.908717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50720 (local address 127.29.84.130:38709)
0806 00:46:27.908840 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:27.908846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:27.908861 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:46:27.909034 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:27.909036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:27.909085 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:27.909192 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:46:27.909416 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.909905 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.910725 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:27.910939 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:27.911051 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:27.911371 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:46:27.911748 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:46:27.911878 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:27.911936 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 00:46:27.911984 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:46:27.917290 19231 raft_consensus.cc:1261] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Refusing update from remote peer 18c0c79f06004794aa77401629406271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:27.917295 19614 raft_consensus.cc:1261] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [term 1 FOLLOWER]: Refusing update from remote peer 18c0c79f06004794aa77401629406271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:27.917948 19708 consensus_queue.cc:1035] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:46:27.918165 19714 consensus_queue.cc:1035] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:46:27.924487 19719 mvcc.cc:205] Tried to move back new op lower bound from 7315362561702653952 to 7315362561574465536. Current Snapshot: MvccSnapshot[applied={T|T < 7315362561702653952}]
I20260806 00:46:27.926561 19718 mvcc.cc:205] Tried to move back new op lower bound from 7315362561702653952 to 7315362561574465536. Current Snapshot: MvccSnapshot[applied={T|T < 7315362561702653952}]
I20260806 00:46:27.932473 19038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:46:27.932641 19038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:46:27.935108 19038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Sending DeleteTablet for 3 replicas of tablet 629ef869d8f849feb7b926b86b28ad0d
I20260806 00:46:27.935739 19398 tablet_service.cc:1514] Processing DeleteTablet for tablet 629ef869d8f849feb7b926b86b28ad0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:27 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:50718
I20260806 00:46:27.935796 19584 tablet_service.cc:1514] Processing DeleteTablet for tablet 629ef869d8f849feb7b926b86b28ad0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:27 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:60646
I20260806 00:46:27.936225 19729 tablet_replica.cc:330] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: stopping tablet replica
I20260806 00:46:27.936213 19204 tablet_service.cc:1514] Processing DeleteTablet for tablet 629ef869d8f849feb7b926b86b28ad0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:27 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:54702
I20260806 00:46:27.936268 19730 tablet_replica.cc:330] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: stopping tablet replica
I20260806 00:46:27.936566 19730 raft_consensus.cc:2229] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:46:27.936617 19729 raft_consensus.cc:2229] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:46:27.936884 19730 raft_consensus.cc:2258] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:46:27.936919 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 19095
I20260806 00:46:27.937320 19729 raft_consensus.cc:2258] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:46:27.944432 19729 ts_tablet_manager.cc:1905] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:46:27.948292 19730 ts_tablet_manager.cc:1905] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:46:27.948410 19729 ts_tablet_manager.cc:1918] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:46:27.948570 19729 log.cc:1196] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/629ef869d8f849feb7b926b86b28ad0d
I20260806 00:46:27.949061 19729 ts_tablet_manager.cc:1939] T 629ef869d8f849feb7b926b86b28ad0d P 18c0c79f06004794aa77401629406271: Deleting consensus metadata
I20260806 00:46:27.950634 19021 catalog_manager.cc:4841] TS 18c0c79f06004794aa77401629406271 (127.29.84.130:38709): tablet 629ef869d8f849feb7b926b86b28ad0d (table test-table [id=3c86150bec2b4293ac74954c8237bdc3]) successfully deleted
I20260806 00:46:27.951210 19730 ts_tablet_manager.cc:1918] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:46:27.951318 19730 log.cc:1196] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/629ef869d8f849feb7b926b86b28ad0d
I20260806 00:46:27.951663 19730 ts_tablet_manager.cc:1939] T 629ef869d8f849feb7b926b86b28ad0d P ff89fec6438e4c6aab562610f89a3c3c: Deleting consensus metadata
I20260806 00:46:27.952762 19019 catalog_manager.cc:4841] TS ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279): tablet 629ef869d8f849feb7b926b86b28ad0d (table test-table [id=3c86150bec2b4293ac74954c8237bdc3]) successfully deleted
W20260806 00:46:27.956663 19023 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:46747 (error 108)
I20260806 00:46:27.957252 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 19301
I20260806 00:46:27.958071 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:27.957405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:46747 (local address 127.0.0.1:54718)
0806 00:46:27.957948 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:27.958008 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:46747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449}
W20260806 00:46:27.958297 19023 catalog_manager.cc:4584] TS bdfd588904954ef7af4c31cc15c1f9d8 (127.29.84.129:46747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 629ef869d8f849feb7b926b86b28ad0d: Network error: Client connection negotiation failed: client connection to 127.29.84.129:46747: connect: Connection refused (error 111)
I20260806 00:46:27.967661 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 19494
I20260806 00:46:27.976615 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 18994
I20260806 00:46:27.988561 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40379
--webserver_interface=127.29.84.190
--webserver_port=37995
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:40379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:28.093474 19734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:28.093727 19734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:28.093807 19734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:28.097582 19734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:28.097653 19734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:28.097687 19734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:28.097759 19734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:28.097810 19734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:28.102736 19734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:40379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=37995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:28.103689 19734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:28.104622 19734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:28.112447 19740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:28.112443 19743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:28.112694 19741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:28.113013 19734 server_base.cc:1034] running on GCE node
I20260806 00:46:28.113525 19734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:28.114581 19734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:28.115767 19734 hybrid_clock.cc:648] HybridClock initialized: now 1785977188115757 us; error 25 us; skew 500 ppm
Aug 06 00:46:28 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:28.118304 19734 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:46:28.119033 19734 webserver.cc:460] Webserver started at http://127.29.84.190:37995/ using document root <none> and password file <none>
I20260806 00:46:28.119609 19734 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:28.119673 19734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:28.123123 19734 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:46:28.125119 19755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:28.126355 19734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:28.126478 19734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "20bd538cd4044290b0dc4d9421d31fbd"
format_stamp: "Formatted at 2026-08-06 00:46:25 on dist-test-slave-n326"
server_key: "e167bb4504100f4f00ddd22b10137d0a"
server_key_iv: "4fbb240b0a9250f76072bd1975b24889"
server_key_version: "encryptionkey@0"
I20260806 00:46:28.127122 19734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:28.137866 19734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:28.140364 19734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:28.140513 19734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:28.147238 19734 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:40379
I20260806 00:46:28.147531 19830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:40379 every 8 connection(s)
I20260806 00:46:28.148267 19734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:46:28.152208 19831 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:46:28.153697 19831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Bootstrap starting.
I20260806 00:46:28.155216 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 19734
I20260806 00:46:28.155691 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:46747
--local_ip_for_outbound_sockets=127.29.84.129
--tserver_master_addrs=127.29.84.190:40379
--webserver_port=46297
--webserver_interface=127.29.84.129
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:46:28.170342 19831 log.cc:824] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:28.180982 19831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:46:28.181344 19831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd: Bootstrap complete.
I20260806 00:46:28.185087 19831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:28.185456 19831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20bd538cd4044290b0dc4d9421d31fbd, State: Initialized, Role: FOLLOWER
I20260806 00:46:28.185902 19831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:28.186049 19831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:46:28.186139 19831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:46:28.186267 19831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 2 FOLLOWER]: Advancing to term 3
I20260806 00:46:28.188030 19831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:28.188359 19831 leader_election.cc:302] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20bd538cd4044290b0dc4d9421d31fbd; no voters:
I20260806 00:46:28.188702 19831 leader_election.cc:288] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 00:46:28.188791 19836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 3 FOLLOWER]: Leader election won for term 3
I20260806 00:46:28.188983 19836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [term 3 LEADER]: Becoming Leader. State: Replica: 20bd538cd4044290b0dc4d9421d31fbd, State: Running, Role: LEADER
I20260806 00:46:28.189275 19836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } }
I20260806 00:46:28.189895 19831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:28.190182 19836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20bd538cd4044290b0dc4d9421d31fbd. Latest consensus state: current_term: 3 leader_uuid: "20bd538cd4044290b0dc4d9421d31fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } } }
I20260806 00:46:28.190304 19836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:28.190665 19843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:46:28.190784 19838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "20bd538cd4044290b0dc4d9421d31fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bd538cd4044290b0dc4d9421d31fbd" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40379 } } }
I20260806 00:46:28.190972 19838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20bd538cd4044290b0dc4d9421d31fbd [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:28.195329 19843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:46:28.195952 19843 catalog_manager.cc:1259] Loaded cluster ID: c95f8ce2141e40a3ae5cb480400aff27
I20260806 00:46:28.196009 19843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 00:46:28.197054 19843 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f865a3e69ad google::LogMessage::Fail() at ??:0
@ 0x7f865a3e988a google::LogMessage::SendToLog() at ??:0
@ 0x7f865a3e64de google::LogMessage::Flush() at ??:0
@ 0x7f865a3e7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8664d25fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8664d26d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8664d23579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8664d67640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f865c6383d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f865c638c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f865c63a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f865c629512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f865dc6d6db start_thread at ??:0
@ 0x7f8658ee461f clone at ??:0
W20260806 00:46:28.291577 19833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:28.291882 19833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:28.291946 19833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:28.297914 19833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:28.297995 19833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:28.298079 19833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:46:28.305459 19833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:46747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=46297
--tserver_master_addrs=127.29.84.190:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:28.306710 19833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:28.308004 19833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:28.318719 19893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:28.318886 19833 server_base.cc:1034] running on GCE node
W20260806 00:46:28.318804 19897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:28.318709 19895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:28.319497 19833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:28.320258 19833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:28.321471 19833 hybrid_clock.cc:648] HybridClock initialized: now 1785977188321451 us; error 29 us; skew 500 ppm
Aug 06 00:46:28 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:28.324863 19833 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:46:28.325933 19833 webserver.cc:460] Webserver started at http://127.29.84.129:46297/ using document root <none> and password file <none>
I20260806 00:46:28.326618 19833 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:28.326716 19833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:28.331564 19833 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:46:28.334087 19911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:28.335614 19833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:46:28.335726 19833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "bdfd588904954ef7af4c31cc15c1f9d8"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "4e10e0877c1d72d2b61e44819cb50025"
server_key_iv: "1e773fec96dbae718444bd715ea198bd"
server_key_version: "encryptionkey@0"
I20260806 00:46:28.336390 19833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:28.357720 19833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:28.361130 19833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:28.361310 19833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:28.361994 19833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:28.363260 19920 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:46:28.367187 19833 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:46:28.367256 19833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.001s
I20260806 00:46:28.367313 19833 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:46:28.369333 19833 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:46:28.369393 19833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 00:46:28.369673 19920 tablet_bootstrap.cc:492] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Bootstrap starting.
I20260806 00:46:28.379848 19833 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:46747
I20260806 00:46:28.380125 20080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:46747 every 8 connection(s)
I20260806 00:46:28.380930 19833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:46:28.386453 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 19833
I20260806 00:46:28.386924 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:38709
--local_ip_for_outbound_sockets=127.29.84.130
--tserver_master_addrs=127.29.84.190:40379
--webserver_port=46783
--webserver_interface=127.29.84.130
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:46:28.389540 19920 log.cc:824] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:28.399545 19920 tablet_bootstrap.cc:492] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:46:28.400000 19920 tablet_bootstrap.cc:492] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Bootstrap complete.
I20260806 00:46:28.401610 19920 ts_tablet_manager.cc:1397] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.008s
I20260806 00:46:28.404517 19920 raft_consensus.cc:348] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:28.404999 19920 raft_consensus.cc:729] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd588904954ef7af4c31cc15c1f9d8, State: Initialized, Role: FOLLOWER
I20260806 00:46:28.405480 19920 consensus_queue.cc:260] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:28.406431 19920 ts_tablet_manager.cc:1428] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8: Time spent starting tablet: real 0.005s user 0.003s sys 0.004s
W20260806 00:46:28.432247 20084 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:46:28.383205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:42263)
0806 00:46:28.383727 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:28.383782 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:28.385020 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:28.431556 (+ 46536us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":133,"threads_started":1}
W20260806 00:46:28.432720 20082 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:40379 (0 consecutive failures): Network error: Failed to ping master at 127.29.84.190:40379: Client connection negotiation failed: client connection to 127.29.84.190:40379: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:46:28.433422 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:28.433196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:58803)
0806 00:46:28.433315 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:28.433372 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 00:46:28.434016 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:28.433819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:36495)
0806 00:46:28.433917 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:28.433972 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:46:28.434187 20082 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:46:28.505445 20086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:28.505681 20086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:28.505757 20086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:28.509660 20086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:28.509758 20086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:28.509850 20086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:46:28.514654 20086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:38709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=46783
--tserver_master_addrs=127.29.84.190:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:28.515609 20086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:28.516561 20086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:28.525718 20099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:28.525764 20101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:28.525990 20086 server_base.cc:1034] running on GCE node
W20260806 00:46:28.525753 20103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:28.526383 20086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:28.527007 20086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:28.528239 20086 hybrid_clock.cc:648] HybridClock initialized: now 1785977188528228 us; error 26 us; skew 500 ppm
Aug 06 00:46:28 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:28.530882 20086 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:46:28.531728 20086 webserver.cc:460] Webserver started at http://127.29.84.130:46783/ using document root <none> and password file <none>
I20260806 00:46:28.532246 20086 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:28.532339 20086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:28.535925 20086 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260806 00:46:28.537784 20114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:28.538887 20086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:28.539018 20086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "18c0c79f06004794aa77401629406271"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "9fd4a8505912b45be5158dcbfc82e1d4"
server_key_iv: "429b882800d38d66effb39e2c20579e8"
server_key_version: "encryptionkey@0"
I20260806 00:46:28.539515 20086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:28.565078 20086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:28.567798 20086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:28.567983 20086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:28.568506 20086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:28.569234 20086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:28.569304 20086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:28.569368 20086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:28.569432 20086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:28.577823 20086 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:38709
I20260806 00:46:28.577868 20277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:38709 every 8 connection(s)
I20260806 00:46:28.578706 20086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:46:28.582127 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:28.580619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:57803)
0806 00:46:28.581066 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:28.581505 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":119,"threads_started":1}
W20260806 00:46:28.582526 20278 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:40379 (0 consecutive failures): Network error: Failed to ping master at 127.29.84.190:40379: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
I20260806 00:46:28.583043 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:28.582853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:59103)
0806 00:46:28.582950 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:28.582993 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 00:46:28.583595 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:28.583422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:49851)
0806 00:46:28.583512 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:28.583552 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 00:46:28.583787 20278 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:46:28.587165 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 20086
I20260806 00:46:28.587554 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:46279
--local_ip_for_outbound_sockets=127.29.84.131
--tserver_master_addrs=127.29.84.190:40379
--webserver_port=36611
--webserver_interface=127.29.84.131
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:28.696422 20282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:28.696646 20282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:28.696686 20282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:28.700625 20282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:28.700671 20282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:28.700766 20282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:46:28.705673 20282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:46279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=36611
--tserver_master_addrs=127.29.84.190:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:28.706547 20282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:28.707506 20282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:28.716341 20293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:28.716331 20296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:28.716348 20294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:28.716692 20282 server_base.cc:1034] running on GCE node
I20260806 00:46:28.717036 20282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:28.717584 20282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:28.718742 20282 hybrid_clock.cc:648] HybridClock initialized: now 1785977188718731 us; error 24 us; skew 500 ppm
Aug 06 00:46:28 dist-test-slave-n326 krb5kdc[18201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:28.721511 20282 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:46:28.722302 20282 webserver.cc:460] Webserver started at http://127.29.84.131:36611/ using document root <none> and password file <none>
I20260806 00:46:28.722837 20282 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:28.722926 20282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:28.726384 20282 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:46:28.728370 20307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:28.729529 20282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:46:28.729668 20282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "ff89fec6438e4c6aab562610f89a3c3c"
format_stamp: "Formatted at 2026-08-06 00:46:26 on dist-test-slave-n326"
server_key: "d707e76dd06f9ad382dc8c1e7caa2358"
server_key_iv: "175cec83715799d3b4213d8012a13bd0"
server_key_version: "encryptionkey@0"
I20260806 00:46:28.730176 20282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:28.744110 20282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:28.746762 20282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:28.746963 20282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:28.747561 20282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:28.748342 20282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:28.748415 20282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:28.748529 20282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:28.748584 20282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:28.757405 20282 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:46279
I20260806 00:46:28.757450 20464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:46279 every 8 connection(s)
I20260806 00:46:28.758312 20282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:46:28.761827 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:28.760296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:45011)
0806 00:46:28.760724 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:28.761155 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":109,"threads_started":1}
W20260806 00:46:28.762338 20466 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:40379 (0 consecutive failures): Network error: Failed to ping master at 127.29.84.190:40379: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
I20260806 00:46:28.762948 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:28.762746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:59181)
0806 00:46:28.762850 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:28.762896 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 00:46:28.763566 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:28.763385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:37493)
0806 00:46:28.763481 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:28.763521 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:46:28.763770 20466 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:46:28.765187 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 20282
I20260806 00:46:29.435179 20470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:29.434708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:43065)
0806 00:46:29.434958 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:29.435011 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260806 00:46:29.584590 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:29.584235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:53787)
0806 00:46:29.584451 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:29.584498 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
I20260806 00:46:29.764803 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:29.764333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:52327)
0806 00:46:29.764612 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:29.764677 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260806 00:46:30.179350 20477 raft_consensus.cc:482] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:30.179592 20477 raft_consensus.cc:504] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:30.180694 20477 leader_election.cc:288] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18c0c79f06004794aa77401629406271 (127.29.84.130:38709), ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279)
Aug 06 00:46:30 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.131@KRBTEST.COM
Aug 06 00:46:30 dist-test-slave-n326 krb5kdc[18201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.130@KRBTEST.COM
I20260806 00:46:30.194371 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:30.180858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:38709 (local address 127.29.84.129:34459)
0806 00:46:30.181203 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:30.181219 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:30.181344 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:30.182914 (+ 1570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:30.182919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:30.183495 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:30.183975 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:30.183989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:30.185254 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:30.185259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:30.186173 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:30.186183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:30.187052 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:30.187056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:30.187320 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:30.188061 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:30.188073 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:30.189894 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:30.192274 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:30.192277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:30.192279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:30.192525 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:30.192854 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:30.192857 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:30.192859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:30.192939 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:30.193536 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:30.193539 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:30.193668 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 00:46:30.193851 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:46:30.194197 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":149,"spinlock_wait_cycles":5760,"thread_start_us":72,"threads_started":1}
I20260806 00:46:30.194381 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:30.181121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:37433 (local address 127.29.84.131:46279)
0806 00:46:30.181575 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:30.181596 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:30.181629 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:46:30.182252 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:30.182257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:30.182554 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:30.182865 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:30.184158 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:30.185091 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:30.186326 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:30.186909 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:30.187113 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:30.189765 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:30.189774 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:30.189791 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:30.189826 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:30.191649 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:30.192274 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:30.192280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:30.192286 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:30.192423 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:30.192839 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:30.192842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:30.192844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:30.193264 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:30.193390 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:30.193960 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 00:46:30.194201 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":121,"threads_started":1}
I20260806 00:46:30.194449 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:30.181036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:34459 (local address 127.29.84.130:38709)
0806 00:46:30.181466 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:30.181491 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:30.181530 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:46:30.182217 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:30.182222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:30.182555 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:30.182865 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:30.184158 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:30.185110 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:30.186301 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:30.186911 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:30.187115 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:30.190067 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:30.190072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:30.190082 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:30.190108 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:30.192172 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:30.192629 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:30.192634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:30.192639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:30.192763 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:30.193030 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:30.193032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:30.193034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:30.193424 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:30.193550 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:30.193977 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 00:46:30.194241 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":68,"threads_started":1}
I20260806 00:46:30.195331 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:30.180933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:46279 (local address 127.29.84.129:37433)
0806 00:46:30.181307 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:30.181322 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:30.181435 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:30.182913 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:30.182919 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:30.183495 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:30.183975 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:30.183990 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:30.185258 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:30.185261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:30.186181 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:30.186192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:30.187035 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:30.187038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:30.187320 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:30.187987 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:30.188003 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:30.189537 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:30.191785 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:30.191791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:30.191804 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:30.192142 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:30.192524 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:30.192526 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:30.192527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:30.192719 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:30.193387 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:30.193393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:30.193670 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 00:46:30.193851 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 00:46:30.195223 (+ 1372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":1321,"spinlock_wait_cycles":37632,"thread_start_us":92,"threads_started":1}
W20260806 00:46:30.197635 19913 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
W20260806 00:46:30.198268 19914 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18c0c79f06004794aa77401629406271 (127.29.84.130:38709): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
I20260806 00:46:30.198365 19914 leader_election.cc:302] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdfd588904954ef7af4c31cc15c1f9d8; no voters: 18c0c79f06004794aa77401629406271, ff89fec6438e4c6aab562610f89a3c3c
I20260806 00:46:30.198709 20477 raft_consensus.cc:2726] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:46:30.436118 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:30.435850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:44179)
0806 00:46:30.435981 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:30.436033 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 00:46:30.585723 20483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:30.585336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:35027)
0806 00:46:30.585586 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:30.585633 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":71,"threads_started":1}
I20260806 00:46:30.767273 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:30.766863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:33191)
0806 00:46:30.767144 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:30.767195 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":68,"threads_started":1}
I20260806 00:46:31.437170 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:31.436790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:44257)
0806 00:46:31.437030 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:31.437079 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260806 00:46:31.586854 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:31.586473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:44383)
0806 00:46:31.586712 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:31.586763 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
I20260806 00:46:31.708549 20487 raft_consensus.cc:482] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:31.708681 20487 raft_consensus.cc:504] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:31.709044 20487 leader_election.cc:288] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18c0c79f06004794aa77401629406271 (127.29.84.130:38709), ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279)
W20260806 00:46:31.709848 19913 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
W20260806 00:46:31.709985 19914 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18c0c79f06004794aa77401629406271 (127.29.84.130:38709): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
I20260806 00:46:31.710101 19914 leader_election.cc:302] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdfd588904954ef7af4c31cc15c1f9d8; no voters: 18c0c79f06004794aa77401629406271, ff89fec6438e4c6aab562610f89a3c3c
I20260806 00:46:31.710289 20487 raft_consensus.cc:2726] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:46:31.768323 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:31.767953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:55533)
0806 00:46:31.768183 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:31.768232 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":72,"threads_started":1}
I20260806 00:46:32.438345 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:32.437925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:32931)
0806 00:46:32.438185 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:32.438237 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":80,"threads_started":1}
I20260806 00:46:32.587942 20490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:32.587548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:36615)
0806 00:46:32.587783 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:32.587831 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":81,"threads_started":1}
I20260806 00:46:32.769531 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:32.769124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:56675)
0806 00:46:32.769377 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:32.769447 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
I20260806 00:46:33.439798 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:33.439384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:44217)
0806 00:46:33.439657 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:33.439708 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":73,"threads_started":1}
I20260806 00:46:33.589113 20494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:33.588731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:35845)
0806 00:46:33.588977 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:33.589029 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260806 00:46:33.629484 20498 raft_consensus.cc:482] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:33.629627 20498 raft_consensus.cc:504] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:33.630095 20498 leader_election.cc:288] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18c0c79f06004794aa77401629406271 (127.29.84.130:38709), ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279)
W20260806 00:46:33.630879 19913 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
W20260806 00:46:33.631018 19914 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18c0c79f06004794aa77401629406271 (127.29.84.130:38709): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
I20260806 00:46:33.631160 19914 leader_election.cc:302] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdfd588904954ef7af4c31cc15c1f9d8; no voters: 18c0c79f06004794aa77401629406271, ff89fec6438e4c6aab562610f89a3c3c
I20260806 00:46:33.631318 20498 raft_consensus.cc:2726] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:46:33.770707 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:33.770314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:43837)
0806 00:46:33.770576 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:33.770629 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":87,"threads_started":1}
I20260806 00:46:34.440927 20501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:34.440541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:60909)
0806 00:46:34.440783 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:34.440834 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260806 00:46:34.590160 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:34.589773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:43209)
0806 00:46:34.590021 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:34.590074 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
I20260806 00:46:34.771895 20505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:34.771498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:33697)
0806 00:46:34.771737 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:34.771797 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260806 00:46:35.442205 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:35.441740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:56587)
0806 00:46:35.442046 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:35.442108 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":143,"threads_started":1}
I20260806 00:46:35.567675 20508 raft_consensus.cc:482] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:35.567862 20508 raft_consensus.cc:504] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:35.568338 20508 leader_election.cc:288] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18c0c79f06004794aa77401629406271 (127.29.84.130:38709), ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279)
W20260806 00:46:35.569232 19914 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18c0c79f06004794aa77401629406271 (127.29.84.130:38709): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
W20260806 00:46:35.569329 19913 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
I20260806 00:46:35.569489 19913 leader_election.cc:302] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdfd588904954ef7af4c31cc15c1f9d8; no voters: 18c0c79f06004794aa77401629406271, ff89fec6438e4c6aab562610f89a3c3c
I20260806 00:46:35.569639 20508 raft_consensus.cc:2726] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:46:35.591282 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:35.590815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:41883)
0806 00:46:35.591145 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:35.591199 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":70,"threads_started":1}
I20260806 00:46:35.773180 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:35.772787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:47771)
0806 00:46:35.773053 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:35.773102 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":68,"threads_started":1}
I20260806 00:46:36.443399 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:36.442965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:57507)
0806 00:46:36.443249 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:36.443303 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":128,"threads_started":1}
I20260806 00:46:36.592319 20513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:36.591932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:45199)
0806 00:46:36.592179 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:36.592230 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260806 00:46:36.774374 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:36.773980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:40259)
0806 00:46:36.774239 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:36.774291 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":84,"threads_started":1}
I20260806 00:46:37.444543 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:37.444112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:34193)
0806 00:46:37.444391 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:37.444448 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":75,"threads_started":1}
I20260806 00:46:37.593472 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:37.593089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:51465)
0806 00:46:37.593338 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:37.593389 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260806 00:46:37.775591 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:37.775166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:50303)
0806 00:46:37.775443 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:37.775508 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":84,"threads_started":1}
I20260806 00:46:38.445788 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:38.445372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:43215)
0806 00:46:38.445633 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:38.445694 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260806 00:46:38.594563 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:38.594208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:43993)
0806 00:46:38.594435 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:38.594484 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":69,"threads_started":1}
I20260806 00:46:38.776669 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:38.776276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:43331)
0806 00:46:38.776537 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:38.776586 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260806 00:46:39.447026 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:39.446649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:32825)
0806 00:46:39.446878 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:39.446930 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
I20260806 00:46:39.595814 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:39.595394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:38931)
0806 00:46:39.595675 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:39.595728 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":69,"threads_started":1}
I20260806 00:46:39.777751 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:39.777341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:57205)
0806 00:46:39.777615 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:39.777668 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":94,"threads_started":1}
I20260806 00:46:40.448163 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:40.447782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:40357)
0806 00:46:40.448018 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:40.448070 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":75,"threads_started":1}
I20260806 00:46:40.596894 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:40.596510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:34819)
0806 00:46:40.596755 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:40.596807 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260806 00:46:40.778888 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:40.778465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:37417)
0806 00:46:40.778743 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:40.778798 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":90,"threads_started":1}
I20260806 00:46:41.449249 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:41.448860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:56045)
0806 00:46:41.449101 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:41.449152 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":82,"threads_started":1}
I20260806 00:46:41.597970 20536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:41.597546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:55097)
0806 00:46:41.597826 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:41.597873 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260806 00:46:41.779982 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:41.779621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:57091)
0806 00:46:41.779848 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:41.779898 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
I20260806 00:46:42.450369 20538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:42.449960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:60271)
0806 00:46:42.450219 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:42.450274 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
I20260806 00:46:42.599197 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:42.598681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:37711)
0806 00:46:42.598970 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:42.599025 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":94,"threads_started":1}
I20260806 00:46:42.781080 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:42.780666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:40051)
0806 00:46:42.780911 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:42.780970 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
I20260806 00:46:42.814087 20542 raft_consensus.cc:482] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:42.814245 20542 raft_consensus.cc:504] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18c0c79f06004794aa77401629406271" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38709 } } peers { permanent_uuid: "ff89fec6438e4c6aab562610f89a3c3c" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 46279 } } peers { permanent_uuid: "bdfd588904954ef7af4c31cc15c1f9d8" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 46747 } }
I20260806 00:46:42.814710 20542 leader_election.cc:288] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18c0c79f06004794aa77401629406271 (127.29.84.130:38709), ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279)
W20260806 00:46:42.815529 19913 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff89fec6438e4c6aab562610f89a3c3c (127.29.84.131:46279): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
W20260806 00:46:42.815675 19914 leader_election.cc:341] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18c0c79f06004794aa77401629406271 (127.29.84.130:38709): Not found: Tablet not found: 629ef869d8f849feb7b926b86b28ad0d
I20260806 00:46:42.815804 19914 leader_election.cc:302] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdfd588904954ef7af4c31cc15c1f9d8; no voters: 18c0c79f06004794aa77401629406271, ff89fec6438e4c6aab562610f89a3c3c
I20260806 00:46:42.815977 20542 raft_consensus.cc:2726] T 629ef869d8f849feb7b926b86b28ad0d P bdfd588904954ef7af4c31cc15c1f9d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:46:43.451750 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:43.451365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.129:56997)
0806 00:46:43.451606 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:43.451656 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":71,"threads_started":1}
I20260806 00:46:43.600404 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:43.600022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.130:56637)
0806 00:46:43.600268 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:43.600319 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260806 00:46:43.766566 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 19833
I20260806 00:46:43.778357 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 20086
I20260806 00:46:43.784754 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:43.782432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40379 (local address 127.29.84.131:38817)
0806 00:46:43.784565 (+ 2133us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:43.784630 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:40379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2038,"thread_start_us":249,"threads_started":1}
I20260806 00:46:43.791896 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 20282
2026-08-06T00:46:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19492 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:46:43 dist-test-slave-n326 krb5kdc[20550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:46:43 dist-test-slave-n326 krb5kdc[20550](info): set up 2 sockets
Aug 06 00:46:43 dist-test-slave-n326 krb5kdc[20550](info): commencing operation
krb5kdc: starting...
W20260806 00:46:45.009300 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:46:45 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:46:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:46:45.132297 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38971
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:38971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:45.242280 20570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:45.242543 20570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:45.242614 20570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:45.246512 20570 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 00:46:45.246583 20570 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 00:46:45.246611 20570 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 00:46:45.246644 20570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:46:45.246680 20570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:45.246726 20570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:45.246757 20570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:46:45.246791 20570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:46:45.251772 20570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:38971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:45.252684 20570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:45.253599 20570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:45.262364 20577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:45.262441 20581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:45.262388 20579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:45.262450 20570 server_base.cc:1034] running on GCE node
I20260806 00:46:45.263286 20570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:45.264302 20570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:45.265475 20570 hybrid_clock.cc:648] HybridClock initialized: now 1785977205265460 us; error 29 us; skew 500 ppm
Aug 06 00:46:45 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977205, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:45.268169 20570 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:46:45.269043 20570 webserver.cc:460] Webserver started at http://127.29.84.190:35929/ using document root <none> and password file <none>
I20260806 00:46:45.269572 20570 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:45.269660 20570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:45.269903 20570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:45.271901 20570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "ba41ee36e7bc4a0eb348e9a578c68a95"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "71a8b12c2e849d7a65b39bea6aca7c69"
server_key_iv: "461a4e9c51228ea7e1d86814277b1eb3"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.272454 20570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "ba41ee36e7bc4a0eb348e9a578c68a95"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "71a8b12c2e849d7a65b39bea6aca7c69"
server_key_iv: "461a4e9c51228ea7e1d86814277b1eb3"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.276588 20570 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:46:45.278890 20590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.280055 20570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:46:45.280198 20570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "ba41ee36e7bc4a0eb348e9a578c68a95"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "71a8b12c2e849d7a65b39bea6aca7c69"
server_key_iv: "461a4e9c51228ea7e1d86814277b1eb3"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.280304 20570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:45.297232 20570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:45.299769 20570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:45.299952 20570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:45.306823 20570 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:38971
I20260806 00:46:45.307051 20662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:38971 every 8 connection(s)
I20260806 00:46:45.307770 20570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:46:45.310318 20663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:45.315532 20663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Bootstrap starting.
I20260806 00:46:45.317996 20663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:45.317997 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 20570
I20260806 00:46:45.318135 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:46:45.318471 30034 external_mini_cluster.cc:1428] Setting key 5b829b0604aeb7504f99b1c040e05643
I20260806 00:46:45.318907 20663 log.cc:824] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:45.321066 20663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: No bootstrap required, opened a new log
I20260806 00:46:45.325814 20663 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:46:45.326090 20663 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:45.326180 20663 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba41ee36e7bc4a0eb348e9a578c68a95, State: Initialized, Role: FOLLOWER
I20260806 00:46:45.326678 20663 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:46:45.326865 20663 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:46:45.326965 20663 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:46:45.327111 20663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:45.327984 20663 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:46:45.328322 20663 leader_election.cc:302] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba41ee36e7bc4a0eb348e9a578c68a95; no voters:
Aug 06 00:46:45 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:45.328683 20663 leader_election.cc:288] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:46:45.328795 20670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:46:45.329015 20670 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 1 LEADER]: Becoming Leader. State: Replica: ba41ee36e7bc4a0eb348e9a578c68a95, State: Running, Role: LEADER
I20260806 00:46:45.329361 20670 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:46:45.330070 20663 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:46:45.331132 20671 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } } }
I20260806 00:46:45.331274 20671 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:45.331561 20672 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba41ee36e7bc4a0eb348e9a578c68a95. Latest consensus state: current_term: 1 leader_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } } }
I20260806 00:46:45.331671 20672 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: This master's current role is: LEADER
I20260806 00:46:45.331954 20684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:46:45.335040 20684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:46:45.335453 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:45.319622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.29.84.190:38971)
0806 00:46:45.320059 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:45.320074 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:45.320109 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:46:45.321395 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:45.321421 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:45.321814 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:45.322161 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:45.322792 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.324336 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.325264 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.325947 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.326124 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:45.329170 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:45.329176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:45.329191 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:45.329220 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:45.331494 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:45.332076 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:45.332082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:45.332087 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:45.332241 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:45.332574 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:45.332577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:45.332579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:45.333079 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:45.333272 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:45.334121 (+ 849us) server_negotiation.cc:299] Negotiation successful
0806 00:46:45.334453 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":137,"threads_started":1}
I20260806 00:46:45.340492 20684 catalog_manager.cc:1347] Generated new cluster ID: 1bcf79357ff343bda4ca31c24d0b93b4
I20260806 00:46:45.340556 20684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:46:45.355521 20684 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:46:45.356593 20684 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:46:45.361040 20684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Generated new TSK 0
I20260806 00:46:45.361711 20684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:46:45.393731 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38971
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:45.501691 20704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:45.501955 20704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:45.502028 20704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:45.505877 20704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:45.505956 20704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:45.506042 20704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:46:45.511098 20704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:45.512013 20704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:45.512976 20704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:45.522111 20710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:45.522095 20713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:45.522255 20704 server_base.cc:1034] running on GCE node
W20260806 00:46:45.522421 20711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:45.522859 20704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:45.523504 20704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:45.524696 20704 hybrid_clock.cc:648] HybridClock initialized: now 1785977205524670 us; error 42 us; skew 500 ppm
Aug 06 00:46:45 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977205, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:45.527369 20704 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:46:45.528151 20704 webserver.cc:460] Webserver started at http://127.29.84.129:38073/ using document root <none> and password file <none>
I20260806 00:46:45.528679 20704 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:45.528775 20704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:45.529026 20704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:45.531014 20704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "eb0731dceffc4bb8860ac3e9c39bca0d"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "8e6fb7086d809f59091a675940ca94c4"
server_key_iv: "72e2a4527e01a5c7bec62ef68e02076e"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.531682 20704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "eb0731dceffc4bb8860ac3e9c39bca0d"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "8e6fb7086d809f59091a675940ca94c4"
server_key_iv: "72e2a4527e01a5c7bec62ef68e02076e"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.535236 20704 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:46:45.537452 20728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.538552 20704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:46:45.538683 20704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "eb0731dceffc4bb8860ac3e9c39bca0d"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "8e6fb7086d809f59091a675940ca94c4"
server_key_iv: "72e2a4527e01a5c7bec62ef68e02076e"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.538790 20704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:45.551719 20704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:45.554198 20704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:45.554388 20704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:45.554900 20704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:45.555707 20704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:45.555781 20704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.555846 20704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:45.555896 20704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.564402 20704 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:37061
I20260806 00:46:45.564445 20901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:37061 every 8 connection(s)
I20260806 00:46:45.565204 20704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:46:45.569654 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 20704
I20260806 00:46:45.569788 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:46:45.570113 30034 external_mini_cluster.cc:1428] Setting key a4459d2247aab57323304d736ae0beee
Aug 06 00:46:45 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977205, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:45.579849 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:45.566813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:48465 (local address 127.29.84.190:38971)
0806 00:46:45.566951 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:45.566955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:45.567693 (+ 738us) server_negotiation.cc:407] Connection header received
0806 00:46:45.568873 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:45.568876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:45.568955 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:45.569052 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:45.570448 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.571116 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.572677 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.572916 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.572992 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:45.575723 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:45.575729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:45.575735 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:45.575768 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:45.577562 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:45.578058 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:45.578065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:45.578069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:45.578171 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:45.578516 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:45.578523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:45.578527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:45.578788 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:45.578909 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:45.579514 (+ 605us) server_negotiation.cc:299] Negotiation successful
0806 00:46:45.579646 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:46:45.580505 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:45.567180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:48465)
0806 00:46:45.567565 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:45.567601 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:45.568675 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:45.569203 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:45.569209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:45.569698 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:45.570295 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:45.570315 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.571246 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.571249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:45.572532 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:45.572547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.573067 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.573074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:45.573291 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:45.573945 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:45.573970 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:45.575539 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:45.577686 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:45.577690 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:45.577701 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:45.577945 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:45.578275 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:45.578277 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:45.578278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:45.578405 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:45.578967 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:45.578972 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:45.579245 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 00:46:45.579399 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 00:46:45.579676 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:46:45.581782 20902 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38971
I20260806 00:46:45.582031 20902 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:45.582475 20902 heartbeater.cc:511] Master 127.29.84.190:38971 requested a full tablet report, sending...
I20260806 00:46:45.584146 20612 ts_manager.cc:194] Registered new tserver with Master: eb0731dceffc4bb8860ac3e9c39bca0d (127.29.84.129:37061)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
I20260806 00:46:45.585392 20612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:48465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:46:45.602034 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38971
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:45.709313 20909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:45.709537 20909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:45.709579 20909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:45.713927 20909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:45.713977 20909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:45.714073 20909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:46:45.718945 20909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:45.719853 20909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:45.720731 20909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:45.729992 20915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:45.730007 20917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:45.730147 20909 server_base.cc:1034] running on GCE node
W20260806 00:46:45.729993 20914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:45.730620 20909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:45.731367 20909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:45.732574 20909 hybrid_clock.cc:648] HybridClock initialized: now 1785977205732560 us; error 33 us; skew 500 ppm
Aug 06 00:46:45 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977205, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:45.735213 20909 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:46:45.735952 20909 webserver.cc:460] Webserver started at http://127.29.84.130:46149/ using document root <none> and password file <none>
I20260806 00:46:45.736416 20909 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:45.736474 20909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:45.736625 20909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:45.738365 20909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "c46ef9b66b994566ac6f137bf9cf0a70"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "5e39a7bca97c3ff312552e2b738976a3"
server_key_iv: "bb3ba89c54407fa4585595b603009327"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.738848 20909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "c46ef9b66b994566ac6f137bf9cf0a70"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "5e39a7bca97c3ff312552e2b738976a3"
server_key_iv: "bb3ba89c54407fa4585595b603009327"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.742352 20909 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:46:45.744719 20927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.745846 20909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:46:45.745970 20909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "c46ef9b66b994566ac6f137bf9cf0a70"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "5e39a7bca97c3ff312552e2b738976a3"
server_key_iv: "bb3ba89c54407fa4585595b603009327"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.746068 20909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:45.765601 20909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:45.768115 20909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:45.768312 20909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:45.768865 20909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:45.769651 20909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:45.769698 20909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.769769 20909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:45.769811 20909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.778715 20909 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:46439
I20260806 00:46:45.778754 21088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:46439 every 8 connection(s)
I20260806 00:46:45.779573 20909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:46:45.787940 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 20909
I20260806 00:46:45.788094 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:46:45.788447 30034 external_mini_cluster.cc:1428] Setting key 74138d96835615d9387f040159a35c89
Aug 06 00:46:45 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977205, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:45.793303 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:45.781220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:53693 (local address 127.29.84.190:38971)
0806 00:46:45.781364 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:45.781369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:45.782622 (+ 1253us) server_negotiation.cc:407] Connection header received
0806 00:46:45.783547 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:45.783550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:45.783664 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:45.783741 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:45.784892 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.785426 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.786483 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.786700 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.786782 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:45.789464 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:45.789467 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:45.789469 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:45.789492 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:45.791262 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:45.791782 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:45.791788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:45.791793 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:45.791890 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:45.792253 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:45.792256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:45.792258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:45.792444 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:45.792559 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:45.793008 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 00:46:45.793125 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:46:45.794021 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:45.781636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:53693)
0806 00:46:45.782470 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:45.782515 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:45.783308 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:45.783907 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:45.783912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:45.784233 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:45.784760 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:45.784773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.785545 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.785547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:45.786363 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:45.786372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.786833 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:45.786836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:45.786976 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:45.787566 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:45.787581 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:45.789313 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:45.791391 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:45.791396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:45.791430 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:45.791676 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:45.791987 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:45.791989 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:45.791990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:45.792119 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:45.792558 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:45.792564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:45.792775 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 00:46:45.792962 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 00:46:45.793250 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":533,"threads_started":1}
I20260806 00:46:45.795113 21089 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38971
I20260806 00:46:45.795346 21089 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:45.795784 21089 heartbeater.cc:511] Master 127.29.84.190:38971 requested a full tablet report, sending...
I20260806 00:46:45.796782 20612 ts_manager.cc:194] Registered new tserver with Master: c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439)
I20260806 00:46:45.797329 20612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:53693
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:46:45.818557 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38971
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:46:45.925475 21094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:46:45.925676 21094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:46:45.925715 21094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:46:45.929340 21094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:46:45.929381 21094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:46:45.929437 21094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:46:45.934408 21094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:46:45.935168 21094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:46:45.935998 21094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:46:45.945092 21103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:45.945140 21106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:46:45.945153 21101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:46:45.945433 21094 server_base.cc:1034] running on GCE node
I20260806 00:46:45.945828 21094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:46:45.946475 21094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:46:45.947669 21094 hybrid_clock.cc:648] HybridClock initialized: now 1785977205947639 us; error 44 us; skew 500 ppm
Aug 06 00:46:45 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977205, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:46:45.950516 21094 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:46:45.951397 21094 webserver.cc:460] Webserver started at http://127.29.84.131:40469/ using document root <none> and password file <none>
I20260806 00:46:45.951929 21094 fs_manager.cc:359] Metadata directory not provided
I20260806 00:46:45.952026 21094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:46:45.952255 21094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:46:45.954102 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "26700deaed384233a392d117c0ce3225"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "9c24261d2d0aec00b6df0605bda2d80f"
server_key_iv: "76cbaf7bbc32abb8f1e30662e8eaaa33"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.954654 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "26700deaed384233a392d117c0ce3225"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "9c24261d2d0aec00b6df0605bda2d80f"
server_key_iv: "76cbaf7bbc32abb8f1e30662e8eaaa33"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.957998 21094 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:46:45.960209 21115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.961351 21094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:46:45.961508 21094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "26700deaed384233a392d117c0ce3225"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "9c24261d2d0aec00b6df0605bda2d80f"
server_key_iv: "76cbaf7bbc32abb8f1e30662e8eaaa33"
server_key_version: "encryptionkey@0"
I20260806 00:46:45.961620 21094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:46:45.979578 21094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:46:45.982614 21094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:46:45.982807 21094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:46:45.983467 21094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:46:45.984261 21094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:46:45.984339 21094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.984407 21094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:46:45.984440 21094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:46:45.992981 21094 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:34323
I20260806 00:46:45.993007 21276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:34323 every 8 connection(s)
I20260806 00:46:45.993762 21094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:46:45.994478 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 21094
I20260806 00:46:45.994609 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:46:45.994894 30034 external_mini_cluster.cc:1428] Setting key b60e0c370720c62a9cf52c2f9788f225
Aug 06 00:46:46 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977205, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:46:46.007190 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:45.995637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:52293 (local address 127.29.84.190:38971)
0806 00:46:45.995755 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:45.995759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:45.996713 (+ 954us) server_negotiation.cc:407] Connection header received
0806 00:46:45.998103 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:45.998106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:45.998163 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:45.998238 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:45.999346 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:45.999908 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.001046 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.001277 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.001414 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.003785 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:46.003789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:46.003792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:46.003825 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:46.005121 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:46.005634 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:46.005638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:46.005640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:46.005726 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:46.006081 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:46.006085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:46.006087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:46.006268 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:46.006385 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:46.006853 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 00:46:46.006990 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 00:46:46.008165 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:45.996048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:52293)
0806 00:46:45.996573 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:45.996612 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:45.997893 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:45.998389 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:45.998395 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:45.998713 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:46:45.999194 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:45.999207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.000063 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.000066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.000904 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.000913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.001449 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.001453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.001626 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.002099 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:46:46.002115 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:46:46.003620 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:46:46.005246 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:46.005252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:46.005265 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:46.005523 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:46.005826 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:46:46.005830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:46:46.005832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:46:46.005970 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:46:46.006384 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:46:46.006391 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:46:46.006614 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 00:46:46.006809 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 00:46:46.007159 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":141,"threads_started":1}
I20260806 00:46:46.009423 21277 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38971
I20260806 00:46:46.009658 21277 heartbeater.cc:464] Registering TS with master...
I20260806 00:46:46.010154 21277 heartbeater.cc:511] Master 127.29.84.190:38971 requested a full tablet report, sending...
I20260806 00:46:46.011246 20612 ts_manager.cc:194] Registered new tserver with Master: 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323)
I20260806 00:46:46.011885 20612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:52293
I20260806 00:46:46.020701 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:46:46.030247 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.022698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.29.84.190:38971)
0806 00:46:46.022840 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:46.022846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:46.023100 (+ 254us) server_negotiation.cc:407] Connection header received
0806 00:46:46.023254 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:46.023257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:46.023306 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:46.023395 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:46:46.024056 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.024553 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.025397 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.025635 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.025740 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.027009 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:46:46.027012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:46:46.027015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:46:46.027039 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:46:46.028550 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:46.028989 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:46.028994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:46.028997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:46.029078 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:46:46.029363 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:46:46.029366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:46:46.029368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:46:46.029575 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:46:46.029719 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:46.029976 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 00:46:46.030085 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:46:46.033107 20612 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:46:46.035979 20612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:46:46.050136 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.044158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:34323 (local address 127.0.0.1:55310)
0806 00:46:46.044519 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:46.044572 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:46.044691 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:46.045270 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:46.045275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:46.045301 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:46.045471 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.045484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.047188 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.047193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.048347 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.048359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.049574 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.049578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.049706 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.049748 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 00:46:46.049860 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 00:46:46.049980 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260806 00:46:46.050690 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.044609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:37061 (local address 127.0.0.1:59736)
0806 00:46:46.044943 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:46.044967 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:46.045047 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:46.045802 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:46.045807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:46.045826 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:46.046038 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.046049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.047761 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.047764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.049071 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.049083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.050252 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.050258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.050393 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.050419 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:46:46.050497 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 00:46:46.050561 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":61,"thread_start_us":93,"threads_started":1}
I20260806 00:46:46.050741 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.044344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.29.84.131:34323)
0806 00:46:46.044757 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:46.044771 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:46.044798 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:46:46.044873 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:46.044878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:46.045090 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:46.045642 (+ 552us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:46.045787 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.046967 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.048508 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.049411 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.049554 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.049722 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:46.050432 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 00:46:46.050539 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":132,"threads_started":1}
I20260806 00:46:46.050865 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.044863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46439 (local address 127.0.0.1:38346)
0806 00:46:46.046062 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:46.046077 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:46.046174 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:46.046716 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:46.046719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:46.046741 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:46.046826 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.046832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.048195 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.048202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.049311 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.049322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.050455 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.050462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.050552 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.050573 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:46:46.050645 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 00:46:46.050712 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"mutex_wait_us":6,"thread_start_us":110,"threads_started":1}
I20260806 00:46:46.051339 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.045018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.29.84.129:37061)
0806 00:46:46.045314 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:46.045332 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:46.045355 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:46:46.045430 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:46.045434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:46.045646 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:46.046215 (+ 569us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:46.046380 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.047597 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.049216 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.050117 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.050227 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.050377 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:46.051042 (+ 665us) server_negotiation.cc:299] Negotiation successful
0806 00:46:46.051185 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":61,"threads_started":1}
I20260806 00:46:46.051477 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.045303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.29.84.130:46439)
0806 00:46:46.045774 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:46.045785 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:46.046272 (+ 487us) server_negotiation.cc:407] Connection header received
0806 00:46:46.046354 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:46.046359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:46.046538 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:46.046705 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:46.046967 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.048077 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.049465 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.050228 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.050317 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.050428 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:46.051196 (+ 768us) server_negotiation.cc:299] Negotiation successful
0806 00:46:46.051307 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":175,"threads_started":1}
I20260806 00:46:46.054049 20992 tablet_service.cc:1467] Processing CreateTablet for tablet d3b260bee964456796be224647760552 (DEFAULT_TABLE table=test-table [id=b9c9687e3c4f4f758ecb8deb9b2d2079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:46.054273 21186 tablet_service.cc:1467] Processing CreateTablet for tablet d3b260bee964456796be224647760552 (DEFAULT_TABLE table=test-table [id=b9c9687e3c4f4f758ecb8deb9b2d2079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:46.054273 20796 tablet_service.cc:1467] Processing CreateTablet for tablet d3b260bee964456796be224647760552 (DEFAULT_TABLE table=test-table [id=b9c9687e3c4f4f758ecb8deb9b2d2079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:46:46.055032 20992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b260bee964456796be224647760552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:46.055405 20796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b260bee964456796be224647760552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:46.055425 21186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b260bee964456796be224647760552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:46:46.061025 21297 tablet_bootstrap.cc:492] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: Bootstrap starting.
I20260806 00:46:46.062662 21299 tablet_bootstrap.cc:492] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Bootstrap starting.
I20260806 00:46:46.063062 21304 tablet_bootstrap.cc:492] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: Bootstrap starting.
I20260806 00:46:46.063792 21297 tablet_bootstrap.cc:654] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:46.064677 21297 log.cc:824] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:46.065655 21304 tablet_bootstrap.cc:654] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:46.066586 21304 log.cc:824] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:46.066905 21297 tablet_bootstrap.cc:492] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: No bootstrap required, opened a new log
I20260806 00:46:46.067157 21297 ts_tablet_manager.cc:1397] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:46:46.067766 21299 tablet_bootstrap.cc:654] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Neither blocks nor log segments found. Creating new log.
I20260806 00:46:46.068631 21304 tablet_bootstrap.cc:492] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: No bootstrap required, opened a new log
I20260806 00:46:46.068820 21304 ts_tablet_manager.cc:1397] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 00:46:46.068923 21299 log.cc:824] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:46:46.070720 21297 raft_consensus.cc:348] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:46:46.070923 21299 tablet_bootstrap.cc:492] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: No bootstrap required, opened a new log
I20260806 00:46:46.070981 21297 raft_consensus.cc:374] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:46.071062 21297 raft_consensus.cc:729] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c46ef9b66b994566ac6f137bf9cf0a70, State: Initialized, Role: FOLLOWER
I20260806 00:46:46.071128 21299 ts_tablet_manager.cc:1397] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 00:46:46.071592 21297 consensus_queue.cc:260] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:46:46.072278 21089 heartbeater.cc:503] Master 127.29.84.190:38971 was elected leader, sending a full tablet report...
I20260806 00:46:46.072712 21297 ts_tablet_manager.cc:1428] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:46:46.073654 21304 raft_consensus.cc:348] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:46:46.073913 21304 raft_consensus.cc:374] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:46.074019 21304 raft_consensus.cc:729] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26700deaed384233a392d117c0ce3225, State: Initialized, Role: FOLLOWER
I20260806 00:46:46.074422 21304 consensus_queue.cc:260] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:46:46.075140 21277 heartbeater.cc:503] Master 127.29.84.190:38971 was elected leader, sending a full tablet report...
I20260806 00:46:46.075380 21304 ts_tablet_manager.cc:1428] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260806 00:46:46.075960 21299 raft_consensus.cc:348] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:46:46.076252 21299 raft_consensus.cc:374] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:46:46.076344 21299 raft_consensus.cc:729] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0731dceffc4bb8860ac3e9c39bca0d, State: Initialized, Role: FOLLOWER
I20260806 00:46:46.076845 21299 consensus_queue.cc:260] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:46:46.077585 20902 heartbeater.cc:503] Master 127.29.84.190:38971 was elected leader, sending a full tablet report...
I20260806 00:46:46.077842 21299 ts_tablet_manager.cc:1428] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:46:46.203351 21313 raft_consensus.cc:482] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:46:46.203632 21313 raft_consensus.cc:504] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:46:46.205171 21313 leader_election.cc:288] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323), c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439)
I20260806 00:46:46.210100 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.205792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:36641 (local address 127.29.84.131:34323)
0806 00:46:46.205905 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:46.205910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:46.205955 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:46:46.206009 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:46.206011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:46.206076 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:46.206155 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:46.206499 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.207278 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.208920 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.209479 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.209608 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.209662 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:46.209882 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 00:46:46.209956 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 00:46:46.210122 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.205473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:34323 (local address 127.29.84.129:36641)
0806 00:46:46.205760 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:46.205780 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:46.205892 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:46.206207 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:46.206210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:46.206234 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:46.206370 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.206379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.207418 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.207420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.208784 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.208796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.209620 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.209622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.209704 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.209762 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 00:46:46.209848 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:46:46.209941 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260806 00:46:46.210259 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.206339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:48583 (local address 127.29.84.130:46439)
0806 00:46:46.206499 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:46.206503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:46.206516 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:46:46.206564 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:46.206567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:46.206613 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:46.206690 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:46:46.206947 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.207612 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.209119 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.209678 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.209782 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.209838 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:46.210059 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 00:46:46.210130 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:46:46.210522 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.205898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46439 (local address 127.29.84.129:48583)
0806 00:46:46.206223 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:46.206239 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:46:46.206321 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:46:46.206720 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:46:46.206722 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:46:46.206739 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:46:46.206831 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.206835 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.207724 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.207727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.209004 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:46:46.209012 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.209793 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.209800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:46:46.209925 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.209946 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:46:46.210021 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:46:46.210080 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":26,"thread_start_us":127,"threads_started":1}
I20260806 00:46:46.210906 21214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b260bee964456796be224647760552" candidate_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26700deaed384233a392d117c0ce3225" is_pre_election: true
I20260806 00:46:46.211138 21020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b260bee964456796be224647760552" candidate_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" is_pre_election: true
I20260806 00:46:46.211239 21214 raft_consensus.cc:2454] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0731dceffc4bb8860ac3e9c39bca0d in term 0.
I20260806 00:46:46.211454 21020 raft_consensus.cc:2454] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0731dceffc4bb8860ac3e9c39bca0d in term 0.
I20260806 00:46:46.211841 20730 leader_election.cc:302] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26700deaed384233a392d117c0ce3225, eb0731dceffc4bb8860ac3e9c39bca0d; no voters:
I20260806 00:46:46.212086 21313 raft_consensus.cc:2781] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:46:46.212188 21313 raft_consensus.cc:482] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:46:46.212248 21313 raft_consensus.cc:3037] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:46.213269 21313 raft_consensus.cc:504] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:46:46.213587 21313 leader_election.cc:288] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 1 election: Requested vote from peers 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323), c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439)
I20260806 00:46:46.214030 21020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b260bee964456796be224647760552" candidate_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c46ef9b66b994566ac6f137bf9cf0a70"
I20260806 00:46:46.214071 21214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b260bee964456796be224647760552" candidate_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26700deaed384233a392d117c0ce3225"
I20260806 00:46:46.214166 21020 raft_consensus.cc:3037] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:46.214227 21214 raft_consensus.cc:3037] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:46:46.215250 21020 raft_consensus.cc:2454] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0731dceffc4bb8860ac3e9c39bca0d in term 1.
I20260806 00:46:46.215583 21214 raft_consensus.cc:2454] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0731dceffc4bb8860ac3e9c39bca0d in term 1.
I20260806 00:46:46.215579 20730 leader_election.cc:302] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c46ef9b66b994566ac6f137bf9cf0a70, eb0731dceffc4bb8860ac3e9c39bca0d; no voters:
I20260806 00:46:46.215777 21313 raft_consensus.cc:2781] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:46:46.216220 21313 raft_consensus.cc:686] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 LEADER]: Becoming Leader. State: Replica: eb0731dceffc4bb8860ac3e9c39bca0d, State: Running, Role: LEADER
I20260806 00:46:46.216588 21313 consensus_queue.cc:237] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:46:46.219678 20612 catalog_manager.cc:5495] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d reported cstate change: term changed from 0 to 1, leader changed from <none> to eb0731dceffc4bb8860ac3e9c39bca0d (127.29.84.129). New cstate: current_term: 1 leader_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } health_report { overall_health: HEALTHY } } }
I20260806 00:46:46.236166 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.232694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.29.84.129:37061)
0806 00:46:46.232808 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 00:46:46.232813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:46:46.232867 (+ 54us) server_negotiation.cc:407] Connection header received
0806 00:46:46.233057 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:46:46.233060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:46:46.233124 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:46:46.233232 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:46:46.233473 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.233941 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.234789 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:46:46.234995 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:46:46.235145 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:46:46.235450 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:46:46.235822 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:46:46.235954 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:46:46.235996 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 00:46:46.236046 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:46:46.241695 21214 raft_consensus.cc:1261] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [term 1 FOLLOWER]: Refusing update from remote peer eb0731dceffc4bb8860ac3e9c39bca0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:46.241695 21020 raft_consensus.cc:1261] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [term 1 FOLLOWER]: Refusing update from remote peer eb0731dceffc4bb8860ac3e9c39bca0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:46:46.242357 21313 consensus_queue.cc:1035] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:46:46.242525 21317 consensus_queue.cc:1035] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:46:46.244729 21278 tablet.cc:2369] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:46:46.249189 21321 mvcc.cc:205] Tried to move back new op lower bound from 7315362636758626304 to 7315362636665593856. Current Snapshot: MvccSnapshot[applied={T|T < 7315362636758626304}]
I20260806 00:46:46.250332 21322 mvcc.cc:205] Tried to move back new op lower bound from 7315362636758626304 to 7315362636665593856. Current Snapshot: MvccSnapshot[applied={T|T < 7315362636758626304}]
I20260806 00:46:46.251240 21323 mvcc.cc:205] Tried to move back new op lower bound from 7315362636758626304 to 7315362636665593856. Current Snapshot: MvccSnapshot[applied={T|T < 7315362636758626304}]
I20260806 00:46:46.258668 20612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:46:46.258817 20612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:46:46.261132 20612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Sending DeleteTablet for 3 replicas of tablet d3b260bee964456796be224647760552
I20260806 00:46:46.261937 21186 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b260bee964456796be224647760552 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:46 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:55310
I20260806 00:46:46.261968 20796 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b260bee964456796be224647760552 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:46 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:59736
I20260806 00:46:46.262105 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 20704
I20260806 00:46:46.262365 20992 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b260bee964456796be224647760552 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:46 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:38346
I20260806 00:46:46.270684 21333 tablet_replica.cc:330] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: stopping tablet replica
I20260806 00:46:46.270963 21333 raft_consensus.cc:2229] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:46:46.271236 21334 tablet_replica.cc:330] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: stopping tablet replica
I20260806 00:46:46.271310 21333 raft_consensus.cc:2258] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:46:46.271544 21334 raft_consensus.cc:2229] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:46:46.271843 21334 raft_consensus.cc:2258] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:46:46.272794 21333 ts_tablet_manager.cc:1905] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:46:46.273051 21334 ts_tablet_manager.cc:1905] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:46:46.275132 21334 ts_tablet_manager.cc:1918] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:46:46.275239 21334 log.cc:1196] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/d3b260bee964456796be224647760552
I20260806 00:46:46.275547 21334 ts_tablet_manager.cc:1939] T d3b260bee964456796be224647760552 P c46ef9b66b994566ac6f137bf9cf0a70: Deleting consensus metadata
I20260806 00:46:46.276273 21333 ts_tablet_manager.cc:1918] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:46:46.276381 21333 log.cc:1196] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/d3b260bee964456796be224647760552
I20260806 00:46:46.276635 20592 catalog_manager.cc:4841] TS c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439): tablet d3b260bee964456796be224647760552 (table test-table [id=b9c9687e3c4f4f758ecb8deb9b2d2079]) successfully deleted
I20260806 00:46:46.276739 21333 ts_tablet_manager.cc:1939] T d3b260bee964456796be224647760552 P 26700deaed384233a392d117c0ce3225: Deleting consensus metadata
I20260806 00:46:46.277896 20592 catalog_manager.cc:4841] TS 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323): tablet d3b260bee964456796be224647760552 (table test-table [id=b9c9687e3c4f4f758ecb8deb9b2d2079]) successfully deleted
W20260806 00:46:46.278004 20591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:37061 (error 108)
I20260806 00:46:46.278467 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 20909
I20260806 00:46:46.281913 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:46:46.278694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:37061 (local address 127.0.0.1:59756)
0806 00:46:46.281777 (+ 3083us) negotiation.cc:107] Waiting for socket to connect
0806 00:46:46.281838 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2995}
W20260806 00:46:46.282068 20591 catalog_manager.cc:4584] TS eb0731dceffc4bb8860ac3e9c39bca0d (127.29.84.129:37061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d3b260bee964456796be224647760552: Network error: Client connection negotiation failed: client connection to 127.29.84.129:37061: connect: Connection refused (error 111)
I20260806 00:46:46.288515 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 21094
I20260806 00:46:46.307345 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 20570
I20260806 00:47:06.319768 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38971
--webserver_interface=127.29.84.190
--webserver_port=35929
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:38971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:06.429442 21337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:06.429737 21337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:06.429809 21337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:06.434782 21337 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 00:47:06.434856 21337 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 00:47:06.434898 21337 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 00:47:06.434929 21337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:06.434966 21337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:06.435019 21337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:06.435050 21337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:06.435096 21337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:06.440658 21337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:38971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=35929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:06.441466 21337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:06.442317 21337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:47:06.454931 21337 server_base.cc:1034] running on GCE node
W20260806 00:47:06.454990 21344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:06.454984 21347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:06.455140 21349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:06.455737 21337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:06.456727 21337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:06.457906 21337 hybrid_clock.cc:648] HybridClock initialized: now 1785977226457890 us; error 32 us; skew 500 ppm
Aug 06 00:47:06 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977226, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:06.464816 21337 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:47:06.465693 21337 webserver.cc:460] Webserver started at http://127.29.84.190:35929/ using document root <none> and password file <none>
I20260806 00:47:06.466239 21337 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:06.466341 21337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:06.469973 21337 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:06.472054 21361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:06.473209 21337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:06.473351 21337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "ba41ee36e7bc4a0eb348e9a578c68a95"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "71a8b12c2e849d7a65b39bea6aca7c69"
server_key_iv: "461a4e9c51228ea7e1d86814277b1eb3"
server_key_version: "encryptionkey@0"
I20260806 00:47:06.473896 21337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:06.488206 21337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:06.490754 21337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:06.490942 21337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:06.497761 21337 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:38971
I20260806 00:47:06.498034 21434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:38971 every 8 connection(s)
I20260806 00:47:06.498736 21337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:06.502736 21435 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:47:06.504336 21435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Bootstrap starting.
I20260806 00:47:06.507337 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 21337
I20260806 00:47:06.507824 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:37061
--local_ip_for_outbound_sockets=127.29.84.129
--tserver_master_addrs=127.29.84.190:38971
--webserver_port=38073
--webserver_interface=127.29.84.129
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:47:06.521883 21435 log.cc:824] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:06.530072 21435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:47:06.530418 21435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Bootstrap complete.
I20260806 00:47:06.534369 21435 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:47:06.534720 21435 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba41ee36e7bc4a0eb348e9a578c68a95, State: Initialized, Role: FOLLOWER
I20260806 00:47:06.535231 21435 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:47:06.535344 21435 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:06.535437 21435 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:06.535548 21435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:47:06.537284 21435 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:47:06.537606 21435 leader_election.cc:302] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba41ee36e7bc4a0eb348e9a578c68a95; no voters:
I20260806 00:47:06.537910 21435 leader_election.cc:288] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:47:06.537972 21440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:47:06.538187 21440 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 2 LEADER]: Becoming Leader. State: Replica: ba41ee36e7bc4a0eb348e9a578c68a95, State: Running, Role: LEADER
I20260806 00:47:06.538467 21440 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:47:06.539105 21435 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:06.539706 21441 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba41ee36e7bc4a0eb348e9a578c68a95. Latest consensus state: current_term: 2 leader_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } } }
I20260806 00:47:06.539829 21441 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:06.540144 21443 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } } }
I20260806 00:47:06.540228 21443 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:06.540500 21447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:06.545051 21447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:06.545689 21447 catalog_manager.cc:1259] Loaded cluster ID: 1bcf79357ff343bda4ca31c24d0b93b4
I20260806 00:47:06.545763 21447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:06.547683 21447 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:06.548252 21447 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Loaded TSK: 0
I20260806 00:47:06.558459 21447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Generated new TSK 1
I20260806 00:47:06.558670 21447 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:47:06.628819 21437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:06.629076 21437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:06.629161 21437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:06.632941 21437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:06.633014 21437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:06.633093 21437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:47:06.638100 21437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:37061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=38073
--tserver_master_addrs=127.29.84.190:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:06.639034 21437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:06.640197 21437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:06.649567 21475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:06.649610 21479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:06.649591 21477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:06.650185 21437 server_base.cc:1034] running on GCE node
I20260806 00:47:06.650617 21437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:06.651333 21437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:06.652504 21437 hybrid_clock.cc:648] HybridClock initialized: now 1785977226652476 us; error 48 us; skew 500 ppm
Aug 06 00:47:06 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977226, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:06.655296 21437 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:47:06.656209 21437 webserver.cc:460] Webserver started at http://127.29.84.129:38073/ using document root <none> and password file <none>
I20260806 00:47:06.656730 21437 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:06.656802 21437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:06.660471 21437 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:47:06.662587 21490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:06.663847 21437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:06.663985 21437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "eb0731dceffc4bb8860ac3e9c39bca0d"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "8e6fb7086d809f59091a675940ca94c4"
server_key_iv: "72e2a4527e01a5c7bec62ef68e02076e"
server_key_version: "encryptionkey@0"
I20260806 00:47:06.664574 21437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:06.685149 21437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:06.688503 21437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:06.688700 21437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:06.689293 21437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:06.690392 21498 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:47:06.694522 21437 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:47:06.694602 21437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 00:47:06.694681 21437 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:47:06.696391 21437 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:47:06.696442 21437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 00:47:06.696756 21498 tablet_bootstrap.cc:492] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Bootstrap starting.
I20260806 00:47:06.706842 21437 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:37061
I20260806 00:47:06.706954 21640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:37061 every 8 connection(s)
I20260806 00:47:06.707814 21437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:47:06.715049 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 21437
I20260806 00:47:06.715544 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:46439
--local_ip_for_outbound_sockets=127.29.84.130
--tserver_master_addrs=127.29.84.190:38971
--webserver_port=46149
--webserver_interface=127.29.84.130
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:47:06.717546 21498 log.cc:824] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:47:06 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977226, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:06.726192 21498 tablet_bootstrap.cc:492] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:47:06.726653 21498 tablet_bootstrap.cc:492] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Bootstrap complete.
I20260806 00:47:06.727339 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:06.709871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:59761 (local address 127.29.84.190:38971)
0806 00:47:06.710279 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:06.710300 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:06.710633 (+ 333us) server_negotiation.cc:407] Connection header received
0806 00:47:06.712091 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:06.712101 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:06.712406 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:06.712673 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:06.714290 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.715608 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.717343 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.718001 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.718177 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:06.721135 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:06.721142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:06.721159 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:06.721190 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:06.723547 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:06.724180 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:06.724186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:06.724192 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:06.724338 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:06.724790 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:06.724793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:06.724795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:06.725263 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:06.725401 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:06.726116 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 00:47:06.726423 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":134,"threads_started":1}
I20260806 00:47:06.727887 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:06.709855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:59761)
0806 00:47:06.710449 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:06.710503 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:06.711704 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:06.712847 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:06.712856 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:06.713342 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:06.714082 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:06.714108 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.715806 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.715812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:06.717194 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:06.717208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.718154 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.718159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:06.718376 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:06.719112 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:06.719141 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:06.720909 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:06.723687 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:06.723693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:06.723705 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:06.724047 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:06.724455 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:06.724460 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:06.724464 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:06.724665 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:06.725402 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:06.725410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:06.725732 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 00:47:06.725966 (+ 234us) client_negotiation.cc:240] Negotiation successful
0806 00:47:06.726444 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":150,"threads_started":1}
I20260806 00:47:06.728163 21498 ts_tablet_manager.cc:1397] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.011s
I20260806 00:47:06.730445 21642 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38971
I20260806 00:47:06.730671 21642 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:06.730540 21498 raft_consensus.cc:348] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:06.731029 21498 raft_consensus.cc:729] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0731dceffc4bb8860ac3e9c39bca0d, State: Initialized, Role: FOLLOWER
I20260806 00:47:06.731225 21642 heartbeater.cc:511] Master 127.29.84.190:38971 requested a full tablet report, sending...
I20260806 00:47:06.731542 21498 consensus_queue.cc:260] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:06.732461 21498 ts_tablet_manager.cc:1428] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 00:47:06.734200 21387 ts_manager.cc:194] Registered new tserver with Master: eb0731dceffc4bb8860ac3e9c39bca0d (127.29.84.129:37061)
I20260806 00:47:06.738004 21387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:59761
I20260806 00:47:06.740520 21642 heartbeater.cc:503] Master 127.29.84.190:38971 was elected leader, sending a full tablet report...
Aug 06 00:47:06 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977226, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.129@KRBTEST.COM
I20260806 00:47:06.747849 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:06.737305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:37061 (local address 127.0.0.1:47664)
0806 00:47:06.737978 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:06.738012 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:06.738228 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:06.738756 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:06.738762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:06.739190 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:06.739306 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:06.739314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.740735 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.740739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:06.741502 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:06.741513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.742249 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.742253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:06.742368 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:06.742953 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:06.742964 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:06.744350 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:06.746108 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:06.746119 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:06.746125 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:06.746350 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:06.746664 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:06.746668 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:06.746670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:06.746755 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:06.747243 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:06.747248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:06.747366 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 00:47:06.747502 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 00:47:06.747682 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":124,"threads_started":1}
I20260806 00:47:06.747884 21652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:06.737352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.29.84.129:37061)
0806 00:47:06.737721 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:06.737731 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:06.738107 (+ 376us) server_negotiation.cc:407] Connection header received
0806 00:47:06.738410 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:06.738416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:06.738622 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:06.738766 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:06.739434 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.740597 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.741626 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.742142 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.742246 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:06.744491 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:06.744501 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:06.744512 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:06.744542 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:06.745996 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:06.746475 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:06.746481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:06.746483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:06.746567 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:06.746851 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:06.746856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:06.746858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:06.747128 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:06.747254 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:06.747595 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 00:47:06.747742 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":111,"threads_started":1}
I20260806 00:47:06.748757 21556 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b260bee964456796be224647760552 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:46:46 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:47664
I20260806 00:47:06.749516 21653 tablet_replica.cc:330] stopping tablet replica
I20260806 00:47:06.749778 21653 raft_consensus.cc:2229] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:06.749995 21653 raft_consensus.cc:2258] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:06.750818 21653 ts_tablet_manager.cc:1905] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:06.754185 21653 ts_tablet_manager.cc:1918] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:06.754266 21653 log.cc:1196] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal/wals/d3b260bee964456796be224647760552
I20260806 00:47:06.754532 21653 ts_tablet_manager.cc:1939] T d3b260bee964456796be224647760552 P eb0731dceffc4bb8860ac3e9c39bca0d: Deleting consensus metadata
I20260806 00:47:06.756027 21362 catalog_manager.cc:4841] TS eb0731dceffc4bb8860ac3e9c39bca0d (127.29.84.129:37061): tablet d3b260bee964456796be224647760552 (table test-table [id=b9c9687e3c4f4f758ecb8deb9b2d2079]) successfully deleted
W20260806 00:47:06.831488 21647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:06.831746 21647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:06.831815 21647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:06.835757 21647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:06.835826 21647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:06.835917 21647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:47:06.840940 21647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:46439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=46149
--tserver_master_addrs=127.29.84.190:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:06.841835 21647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:06.842736 21647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:06.851559 21665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:06.851568 21667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:06.851666 21647 server_base.cc:1034] running on GCE node
W20260806 00:47:06.851541 21659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:06.852108 21647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:06.852669 21647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:06.853824 21647 hybrid_clock.cc:648] HybridClock initialized: now 1785977226853806 us; error 30 us; skew 500 ppm
Aug 06 00:47:06 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977226, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:06.856499 21647 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:47:06.857250 21647 webserver.cc:460] Webserver started at http://127.29.84.130:46149/ using document root <none> and password file <none>
I20260806 00:47:06.857707 21647 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:06.857774 21647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:06.861069 21647 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 00:47:06.862879 21680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:06.864049 21647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 00:47:06.864140 21647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "c46ef9b66b994566ac6f137bf9cf0a70"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "5e39a7bca97c3ff312552e2b738976a3"
server_key_iv: "bb3ba89c54407fa4585595b603009327"
server_key_version: "encryptionkey@0"
I20260806 00:47:06.864579 21647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:06.891974 21647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:06.894457 21647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:06.894588 21647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:06.895035 21647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:06.895723 21647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:06.895768 21647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:06.895803 21647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:06.895823 21647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:06.904307 21647 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:46439
I20260806 00:47:06.904330 21840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:46439 every 8 connection(s)
I20260806 00:47:06.905191 21647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
Aug 06 00:47:06 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977226, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:06.913908 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 21647
I20260806 00:47:06.914340 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:34323
--local_ip_for_outbound_sockets=127.29.84.131
--tserver_master_addrs=127.29.84.190:38971
--webserver_port=40469
--webserver_interface=127.29.84.131
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:47:06.917816 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:06.906864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:33847 (local address 127.29.84.190:38971)
0806 00:47:06.907032 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:06.907037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:06.907770 (+ 733us) server_negotiation.cc:407] Connection header received
0806 00:47:06.908875 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:06.908878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:06.908929 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:06.909022 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:06.910029 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.910542 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.911593 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.911827 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.911929 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:06.914232 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:06.914235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:06.914237 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:06.914260 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:06.915632 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:06.916133 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:06.916141 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:06.916142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:06.916262 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:06.916648 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:06.916652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:06.916654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:06.916888 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:06.916996 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:06.917462 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 00:47:06.917603 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 00:47:06.918327 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:06.907231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:33847)
0806 00:47:06.907632 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:06.907670 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:06.908678 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:06.909180 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:06.909186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:06.909494 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:06.909909 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:06.909921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.910648 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.910650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:06.911489 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:06.911498 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:06.911974 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:06.911976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:06.912111 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:06.912631 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:06.912647 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:06.914066 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:06.915756 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:06.915761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:06.915773 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:06.916027 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:06.916378 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:06.916384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:06.916387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:06.916552 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:06.916992 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:06.916996 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:06.917186 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 00:47:06.917386 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 00:47:06.917606 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":101,"threads_started":1}
I20260806 00:47:06.919426 21842 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38971
I20260806 00:47:06.919652 21842 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:06.920085 21842 heartbeater.cc:511] Master 127.29.84.190:38971 requested a full tablet report, sending...
I20260806 00:47:06.921053 21387 ts_manager.cc:194] Registered new tserver with Master: c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439)
I20260806 00:47:06.921669 21387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:33847
W20260806 00:47:07.022048 21847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:07.022294 21847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:07.022361 21847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:07.026361 21847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:07.026443 21847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:07.026527 21847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:47:07.031236 21847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:34323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=40469
--tserver_master_addrs=127.29.84.190:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:07.032091 21847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:07.032960 21847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:07.042186 21857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:07.042194 21855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:07.042238 21854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:07.042459 21847 server_base.cc:1034] running on GCE node
I20260806 00:47:07.042869 21847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:07.043480 21847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:07.044642 21847 hybrid_clock.cc:648] HybridClock initialized: now 1785977227044628 us; error 25 us; skew 500 ppm
Aug 06 00:47:07 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977227, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:07.047263 21847 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:47:07.048007 21847 webserver.cc:460] Webserver started at http://127.29.84.131:40469/ using document root <none> and password file <none>
I20260806 00:47:07.048535 21847 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:07.048643 21847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:07.052119 21847 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:07.054033 21866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:07.055215 21847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:07.055335 21847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "26700deaed384233a392d117c0ce3225"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "9c24261d2d0aec00b6df0605bda2d80f"
server_key_iv: "76cbaf7bbc32abb8f1e30662e8eaaa33"
server_key_version: "encryptionkey@0"
I20260806 00:47:07.055794 21847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:07.079177 21847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:07.082269 21847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:07.082460 21847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:07.082998 21847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:07.083796 21847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:07.083868 21847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:07.083936 21847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:07.083985 21847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:07.092969 21847 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:34323
I20260806 00:47:07.093015 22028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:34323 every 8 connection(s)
I20260806 00:47:07.093863 21847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:47:07.102250 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 21847
Aug 06 00:47:07 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977227, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:07.106739 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.095528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:37257 (local address 127.29.84.190:38971)
0806 00:47:07.095663 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:07.095666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:07.096474 (+ 808us) server_negotiation.cc:407] Connection header received
0806 00:47:07.097630 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:07.097633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:07.097683 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:07.097758 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:07.098788 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.099340 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.100413 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.100632 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.100713 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.103145 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:07.103149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:07.103152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:07.103180 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:07.104706 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:07.105260 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:07.105263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:07.105265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:07.105361 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:07.105684 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:07.105688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:07.105690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:07.105900 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:07.105994 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:07.106450 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 00:47:07.106557 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":51}
I20260806 00:47:07.107543 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.095906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:37257)
0806 00:47:07.096320 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:07.096359 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:07.097430 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:07.097910 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:07.097916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:07.098229 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:07.098647 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.098659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.099464 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.099466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.100299 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.100308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.100759 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.100762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.100898 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.101429 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:07.101445 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:07.102939 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:07.104819 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:07.104824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:07.104836 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:07.105162 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:07.105458 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:07.105460 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:07.105461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:07.105587 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:07.105997 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:07.106001 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:07.106211 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 00:47:07.106404 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 00:47:07.106695 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":107,"threads_started":1}
I20260806 00:47:07.108596 22030 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38971
I20260806 00:47:07.108785 22030 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:07.109157 22030 heartbeater.cc:511] Master 127.29.84.190:38971 requested a full tablet report, sending...
I20260806 00:47:07.110091 21387 ts_manager.cc:194] Registered new tserver with Master: 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323)
I20260806 00:47:07.110610 21387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:37257
I20260806 00:47:07.112226 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.102985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54362 (local address 127.29.84.190:38971)
0806 00:47:07.103275 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:07.103280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:07.103408 (+ 128us) server_negotiation.cc:407] Connection header received
0806 00:47:07.103568 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:07.103571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:07.103622 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:07.103714 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:07.104451 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.105102 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.106233 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.106548 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.106658 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.108074 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:07.108077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:07.108079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:07.108099 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:07.110607 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:07.111029 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:07.111033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:07.111035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:07.111196 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:07.111479 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:07.111483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:07.111484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:07.111678 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:07.111899 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:07.111976 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 00:47:07.112085 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":120,"threads_started":1}
I20260806 00:47:07.113854 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:47:07.117234 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.114448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.29.84.190:38971)
0806 00:47:07.114560 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:07.114563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:07.114723 (+ 160us) server_negotiation.cc:407] Connection header received
0806 00:47:07.114820 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:07.114822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:07.114905 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:07.114990 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:47:07.115167 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.115601 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.116337 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.116558 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.116632 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.116866 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:47:07.116991 (+ 125us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 00:47:07.117160 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54370: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 00:47:07.117309 22035 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54370: token expired
W20260806 00:47:07.117623 21284 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.84.190:38971)
I20260806 00:47:07.124420 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.117945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.29.84.190:38971)
0806 00:47:07.118050 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:07.118053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:07.118064 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:47:07.118140 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:07.118142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:07.118170 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:07.118230 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:07.118754 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.119246 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.119954 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.120141 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.120211 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.121406 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:07.121409 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:07.121410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:07.121448 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:07.122923 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:07.123346 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:07.123349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:07.123350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:07.123421 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:07.123684 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:07.123687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:07.123688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:07.123856 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:07.123971 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:07.124171 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 00:47:07.124297 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 00:47:07.126394 21387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:47:07.129396 21387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:47:07.136291 21556 tablet_service.cc:1467] Processing CreateTablet for tablet 7856871feea140b8b1bd0e1c5a837800 (DEFAULT_TABLE table=test-table [id=c45be591720d4da090aa10e57e0c5cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:07.136921 21556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7856871feea140b8b1bd0e1c5a837800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:07.139792 21498 tablet_bootstrap.cc:492] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Bootstrap starting.
I20260806 00:47:07.140671 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.135633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:34323 (local address 127.0.0.1:53750)
0806 00:47:07.135933 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:07.135948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:07.136036 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:07.136473 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:07.136475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:07.136493 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:07.136583 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.136590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.137729 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.137735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.139118 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.139129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.140278 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.140281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.140366 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.140402 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 00:47:07.140481 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 00:47:07.140547 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":84,"threads_started":1}
I20260806 00:47:07.140691 21498 tablet_bootstrap.cc:654] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:07.141242 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.135766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.29.84.131:34323)
0806 00:47:07.136058 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:07.136075 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:07.136099 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:47:07.136170 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:07.136174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:07.136329 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:07.136487 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:07.136677 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.137412 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.139243 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.140158 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.140277 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.140416 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:07.140971 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 00:47:07.141059 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":127,"threads_started":1}
I20260806 00:47:07.141431 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.135229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46439 (local address 127.0.0.1:39892)
0806 00:47:07.135377 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:07.135390 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:07.135460 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:07.136689 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:07.136692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:07.136709 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:07.136785 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.136790 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.138310 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.138314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.139414 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.139421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.140943 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.140947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.141117 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.141149 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 00:47:07.141203 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 00:47:07.141282 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 00:47:07.142019 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.135420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39892 (local address 127.29.84.130:46439)
0806 00:47:07.136180 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:07.136191 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:07.136213 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:47:07.136284 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:07.136289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:07.136527 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:07.136930 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:07.137118 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.138136 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.139553 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.140765 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.141602 (+ 837us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.141724 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:07.141764 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 00:47:07.141866 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":137,"threads_started":1}
I20260806 00:47:07.142339 21498 tablet_bootstrap.cc:492] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: No bootstrap required, opened a new log
I20260806 00:47:07.142443 21498 ts_tablet_manager.cc:1397] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 00:47:07.142853 21498 raft_consensus.cc:348] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:07.142976 21498 raft_consensus.cc:374] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:07.143015 21498 raft_consensus.cc:729] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0731dceffc4bb8860ac3e9c39bca0d, State: Initialized, Role: FOLLOWER
I20260806 00:47:07.143213 21498 consensus_queue.cc:260] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:07.143556 21498 ts_tablet_manager.cc:1428] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:47:07.144552 21932 tablet_service.cc:1467] Processing CreateTablet for tablet 7856871feea140b8b1bd0e1c5a837800 (DEFAULT_TABLE table=test-table [id=c45be591720d4da090aa10e57e0c5cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:07.144873 21748 tablet_service.cc:1467] Processing CreateTablet for tablet 7856871feea140b8b1bd0e1c5a837800 (DEFAULT_TABLE table=test-table [id=c45be591720d4da090aa10e57e0c5cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:07.145510 21932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7856871feea140b8b1bd0e1c5a837800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:07.145799 21748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7856871feea140b8b1bd0e1c5a837800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:07.151491 22049 tablet_bootstrap.cc:492] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: Bootstrap starting.
I20260806 00:47:07.152154 22050 tablet_bootstrap.cc:492] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: Bootstrap starting.
I20260806 00:47:07.154086 22049 tablet_bootstrap.cc:654] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:07.154246 22050 tablet_bootstrap.cc:654] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:07.154820 22049 log.cc:824] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:07.154914 22050 log.cc:824] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:07.156602 22049 tablet_bootstrap.cc:492] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: No bootstrap required, opened a new log
I20260806 00:47:07.156705 22050 tablet_bootstrap.cc:492] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: No bootstrap required, opened a new log
I20260806 00:47:07.156814 22049 ts_tablet_manager.cc:1397] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:47:07.156908 22050 ts_tablet_manager.cc:1397] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 00:47:07.159873 22050 raft_consensus.cc:348] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:07.159929 22049 raft_consensus.cc:348] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:07.160100 22050 raft_consensus.cc:374] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:07.160163 22050 raft_consensus.cc:729] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c46ef9b66b994566ac6f137bf9cf0a70, State: Initialized, Role: FOLLOWER
I20260806 00:47:07.160175 22049 raft_consensus.cc:374] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:07.160248 22049 raft_consensus.cc:729] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26700deaed384233a392d117c0ce3225, State: Initialized, Role: FOLLOWER
I20260806 00:47:07.160624 22050 consensus_queue.cc:260] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:07.160641 22049 consensus_queue.cc:260] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:07.161243 21842 heartbeater.cc:503] Master 127.29.84.190:38971 was elected leader, sending a full tablet report...
I20260806 00:47:07.161257 22030 heartbeater.cc:503] Master 127.29.84.190:38971 was elected leader, sending a full tablet report...
I20260806 00:47:07.161654 22050 ts_tablet_manager.cc:1428] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:47:07.161654 22049 ts_tablet_manager.cc:1428] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:47:07.171833 22057 raft_consensus.cc:482] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:07.171995 22057 raft_consensus.cc:504] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:07.172899 22057 leader_election.cc:288] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439), eb0731dceffc4bb8860ac3e9c39bca0d (127.29.84.129:37061)
I20260806 00:47:07.176858 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.173125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:45283 (local address 127.29.84.130:46439)
0806 00:47:07.173252 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:07.173259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:07.173366 (+ 107us) server_negotiation.cc:407] Connection header received
0806 00:47:07.173495 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:07.173498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:07.173549 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:07.173630 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:07.173926 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.174603 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.175751 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.176348 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.176431 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.176466 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:07.176723 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 00:47:07.176776 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 00:47:07.176865 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.173069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46439 (local address 127.29.84.131:45283)
0806 00:47:07.173256 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:07.173273 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:07.173362 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:07.173655 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:07.173659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:07.173677 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:07.173787 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.173794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.174705 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.174708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.175633 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.175642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.176491 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.176495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.176595 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.176617 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:47:07.176685 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 00:47:07.176748 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260806 00:47:07.177428 21773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7856871feea140b8b1bd0e1c5a837800" candidate_uuid: "26700deaed384233a392d117c0ce3225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" is_pre_election: true
I20260806 00:47:07.177518 21652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.173359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:46091 (local address 127.29.84.129:37061)
0806 00:47:07.173465 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:07.173470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:07.173707 (+ 237us) server_negotiation.cc:407] Connection header received
0806 00:47:07.173930 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:07.173933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:07.173989 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:07.174054 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:07.174424 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.175135 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.176275 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.176879 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.177007 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.177141 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:07.177348 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 00:47:07.177411 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 00:47:07.177515 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.173108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:37061 (local address 127.29.84.131:46091)
0806 00:47:07.173627 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:07.173640 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:07.173796 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:07.174187 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:07.174190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:07.174205 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:07.174315 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.174319 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.175239 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.175242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.176156 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:07.176166 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.177135 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.177138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:07.177231 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.177246 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 00:47:07.177329 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 00:47:07.177386 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":131,"threads_started":1}
I20260806 00:47:07.177712 21773 raft_consensus.cc:2454] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26700deaed384233a392d117c0ce3225 in term 0.
I20260806 00:47:07.177982 21580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7856871feea140b8b1bd0e1c5a837800" candidate_uuid: "26700deaed384233a392d117c0ce3225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" is_pre_election: true
I20260806 00:47:07.178190 21871 leader_election.cc:302] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26700deaed384233a392d117c0ce3225, c46ef9b66b994566ac6f137bf9cf0a70; no voters:
I20260806 00:47:07.178223 21580 raft_consensus.cc:2454] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26700deaed384233a392d117c0ce3225 in term 0.
I20260806 00:47:07.178447 22057 raft_consensus.cc:2781] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:47:07.178558 22057 raft_consensus.cc:482] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:47:07.178604 22057 raft_consensus.cc:3037] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:07.179561 22057 raft_consensus.cc:504] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:07.179843 22057 leader_election.cc:288] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [CANDIDATE]: Term 1 election: Requested vote from peers c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439), eb0731dceffc4bb8860ac3e9c39bca0d (127.29.84.129:37061)
I20260806 00:47:07.180202 21580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7856871feea140b8b1bd0e1c5a837800" candidate_uuid: "26700deaed384233a392d117c0ce3225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d"
I20260806 00:47:07.180204 21773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7856871feea140b8b1bd0e1c5a837800" candidate_uuid: "26700deaed384233a392d117c0ce3225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c46ef9b66b994566ac6f137bf9cf0a70"
I20260806 00:47:07.180333 21580 raft_consensus.cc:3037] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:07.180343 21773 raft_consensus.cc:3037] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:07.181257 21773 raft_consensus.cc:2454] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26700deaed384233a392d117c0ce3225 in term 1.
I20260806 00:47:07.181288 21580 raft_consensus.cc:2454] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26700deaed384233a392d117c0ce3225 in term 1.
I20260806 00:47:07.181571 21871 leader_election.cc:302] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26700deaed384233a392d117c0ce3225, c46ef9b66b994566ac6f137bf9cf0a70; no voters:
I20260806 00:47:07.181751 22057 raft_consensus.cc:2781] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:07.181959 22057 raft_consensus.cc:686] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 1 LEADER]: Becoming Leader. State: Replica: 26700deaed384233a392d117c0ce3225, State: Running, Role: LEADER
I20260806 00:47:07.182264 22057 consensus_queue.cc:237] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:07.185343 21387 catalog_manager.cc:5495] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26700deaed384233a392d117c0ce3225 (127.29.84.131). New cstate: current_term: 1 leader_uuid: "26700deaed384233a392d117c0ce3225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } health_report { overall_health: UNKNOWN } } }
I20260806 00:47:07.203832 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:07.200461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53764 (local address 127.29.84.131:34323)
0806 00:47:07.200584 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:07.200590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:07.200609 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:47:07.200795 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:07.200797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:07.200842 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:07.200945 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:47:07.201192 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.201678 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.202494 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:07.202706 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:07.202818 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:07.203136 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:47:07.203458 (+ 322us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:47:07.203588 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:07.203660 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 00:47:07.203708 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:47:07.208726 21773 raft_consensus.cc:1261] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [term 1 FOLLOWER]: Refusing update from remote peer 26700deaed384233a392d117c0ce3225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:07.208739 21580 raft_consensus.cc:1261] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Refusing update from remote peer 26700deaed384233a392d117c0ce3225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 00:47:07.209096 21643 tablet.cc:2369] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:47:07.209348 22059 consensus_queue.cc:1035] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:07.209511 22057 consensus_queue.cc:1035] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:07.214987 22063 mvcc.cc:205] Tried to move back new op lower bound from 7315362722641367040 to 7315362722541154304. Current Snapshot: MvccSnapshot[applied={T|T < 7315362722641367040}]
I20260806 00:47:07.215399 22062 mvcc.cc:205] Tried to move back new op lower bound from 7315362722641367040 to 7315362722541154304. Current Snapshot: MvccSnapshot[applied={T|T < 7315362722641367040}]
I20260806 00:47:07.222642 21387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:07.222798 21387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:07.225095 21387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Sending DeleteTablet for 3 replicas of tablet 7856871feea140b8b1bd0e1c5a837800
I20260806 00:47:07.225771 21556 tablet_service.cc:1514] Processing DeleteTablet for tablet 7856871feea140b8b1bd0e1c5a837800 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:07 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:47664
I20260806 00:47:07.225798 21932 tablet_service.cc:1514] Processing DeleteTablet for tablet 7856871feea140b8b1bd0e1c5a837800 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:07 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:53750
I20260806 00:47:07.225803 21748 tablet_service.cc:1514] Processing DeleteTablet for tablet 7856871feea140b8b1bd0e1c5a837800 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:07 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:39892
I20260806 00:47:07.225955 21653 tablet_replica.cc:330] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: stopping tablet replica
I20260806 00:47:07.226076 21653 raft_consensus.cc:2229] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:07.226188 21653 raft_consensus.cc:2258] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:07.226227 22076 tablet_replica.cc:330] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: stopping tablet replica
I20260806 00:47:07.226643 22076 raft_consensus.cc:2229] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:07.226684 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 21437
I20260806 00:47:07.226912 22076 raft_consensus.cc:2258] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:07.227049 21653 ts_tablet_manager.cc:1905] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:07.237005 22076 ts_tablet_manager.cc:1905] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:07.239464 22075 tablet_replica.cc:330] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: stopping tablet replica
I20260806 00:47:07.239771 22076 ts_tablet_manager.cc:1918] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:07.239912 22076 log.cc:1196] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/7856871feea140b8b1bd0e1c5a837800
I20260806 00:47:07.239867 22075 raft_consensus.cc:2229] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:47:07.240262 22076 ts_tablet_manager.cc:1939] T 7856871feea140b8b1bd0e1c5a837800 P c46ef9b66b994566ac6f137bf9cf0a70: Deleting consensus metadata
I20260806 00:47:07.240414 22075 raft_consensus.cc:2258] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:07.241684 21364 catalog_manager.cc:4841] TS c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439): tablet 7856871feea140b8b1bd0e1c5a837800 (table test-table [id=c45be591720d4da090aa10e57e0c5cdf]) successfully deleted
I20260806 00:47:07.241853 22075 ts_tablet_manager.cc:1905] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 00:47:07.244143 21362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:37061 (error 108)
I20260806 00:47:07.244956 22075 ts_tablet_manager.cc:1918] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:07.245061 22075 log.cc:1196] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/7856871feea140b8b1bd0e1c5a837800
I20260806 00:47:07.245417 22075 ts_tablet_manager.cc:1939] T 7856871feea140b8b1bd0e1c5a837800 P 26700deaed384233a392d117c0ce3225: Deleting consensus metadata
W20260806 00:47:07.245755 21651 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:47:07.245104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:37061 (local address 127.0.0.1:47674)
0806 00:47:07.245316 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:07.245334 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:07.245437 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:07.245665 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:37061: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 00:47:07.245936 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 21647
W20260806 00:47:07.245999 21362 catalog_manager.cc:4584] TS eb0731dceffc4bb8860ac3e9c39bca0d (127.29.84.129:37061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7856871feea140b8b1bd0e1c5a837800: Network error: Client connection negotiation failed: client connection to 127.29.84.129:37061: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:47:07.249423 21364 catalog_manager.cc:4841] TS 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323): tablet 7856871feea140b8b1bd0e1c5a837800 (table test-table [id=c45be591720d4da090aa10e57e0c5cdf]) successfully deleted
I20260806 00:47:07.261940 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 21847
I20260806 00:47:07.270696 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 21337
I20260806 00:47:27.281842 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38971
--webserver_interface=127.29.84.190
--webserver_port=35929
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:38971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:27.391808 22080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:27.392031 22080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:27.392088 22080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:27.395956 22080 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 00:47:27.395999 22080 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 00:47:27.396014 22080 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 00:47:27.396028 22080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:27.396045 22080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:27.396111 22080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:27.396132 22080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:27.396171 22080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:27.401046 22080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:38971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=35929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:27.401961 22080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:27.402889 22080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:27.411190 22097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:27.411190 22092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:27.411231 22090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:27.411231 22080 server_base.cc:1034] running on GCE node
I20260806 00:47:27.411976 22080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:27.413014 22080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:27.414191 22080 hybrid_clock.cc:648] HybridClock initialized: now 1785977247414178 us; error 26 us; skew 500 ppm
Aug 06 00:47:27 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:27.421151 22080 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:47:27.422147 22080 webserver.cc:460] Webserver started at http://127.29.84.190:35929/ using document root <none> and password file <none>
I20260806 00:47:27.422686 22080 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:27.422789 22080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:27.426393 22080 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:47:27.428560 22111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:27.429654 22080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:47:27.429816 22080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "ba41ee36e7bc4a0eb348e9a578c68a95"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "71a8b12c2e849d7a65b39bea6aca7c69"
server_key_iv: "461a4e9c51228ea7e1d86814277b1eb3"
server_key_version: "encryptionkey@0"
I20260806 00:47:27.430325 22080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:27.460224 22080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:27.462944 22080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:27.463143 22080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:27.469923 22080 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:38971
I20260806 00:47:27.470233 22188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:38971 every 8 connection(s)
I20260806 00:47:27.470959 22080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:27.474951 22189 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:47:27.476536 22189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Bootstrap starting.
I20260806 00:47:27.479084 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 22080
I20260806 00:47:27.479610 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:37061
--local_ip_for_outbound_sockets=127.29.84.129
--tserver_master_addrs=127.29.84.190:38971
--webserver_port=38073
--webserver_interface=127.29.84.129
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:47:27.494403 22189 log.cc:824] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:27.505038 22189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:47:27.505383 22189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Bootstrap complete.
I20260806 00:47:27.509162 22189 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:47:27.509518 22189 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba41ee36e7bc4a0eb348e9a578c68a95, State: Initialized, Role: FOLLOWER
I20260806 00:47:27.510056 22189 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:47:27.510172 22189 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:27.510257 22189 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:27.510380 22189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 2 FOLLOWER]: Advancing to term 3
I20260806 00:47:27.512183 22189 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:47:27.512532 22189 leader_election.cc:302] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba41ee36e7bc4a0eb348e9a578c68a95; no voters:
I20260806 00:47:27.512869 22189 leader_election.cc:288] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 00:47:27.512914 22193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 00:47:27.513108 22193 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [term 3 LEADER]: Becoming Leader. State: Replica: ba41ee36e7bc4a0eb348e9a578c68a95, State: Running, Role: LEADER
I20260806 00:47:27.513376 22193 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } }
I20260806 00:47:27.513983 22189 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:27.514230 22194 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba41ee36e7bc4a0eb348e9a578c68a95. Latest consensus state: current_term: 3 leader_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } } }
I20260806 00:47:27.514353 22194 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:27.514281 22195 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba41ee36e7bc4a0eb348e9a578c68a95" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38971 } } }
I20260806 00:47:27.514396 22195 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95 [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:27.514704 22198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:27.519743 22198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:27.520437 22198 catalog_manager.cc:1259] Loaded cluster ID: 1bcf79357ff343bda4ca31c24d0b93b4
I20260806 00:47:27.520515 22198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:27.522483 22198 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:27.523165 22198 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Loaded TSK: 0
I20260806 00:47:27.523252 22198 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ba41ee36e7bc4a0eb348e9a578c68a95: Loaded TSK: 1
F20260806 00:47:27.523808 22198 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fea7d6f79ad google::LogMessage::Fail() at ??:0
@ 0x7fea7d6fa88a google::LogMessage::SendToLog() at ??:0
@ 0x7fea7d6f74de google::LogMessage::Flush() at ??:0
@ 0x7fea7d6f8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fea803bb18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fea803b3f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fea8805c7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fea8803699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fea88037178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fea88078df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fea8809782b std::function<>::operator()() at ??:0
@ 0x7fea88036b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fea88037f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fea88034579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fea88078640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fea7f9493d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fea7f949c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fea7f94b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fea7f93a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fea80f7e6db start_thread at ??:0
@ 0x7fea7c1f561f clone at ??:0
W20260806 00:47:27.607618 22191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:27.607832 22191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:27.607872 22191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:27.612002 22191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:27.612042 22191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:27.612094 22191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:47:27.617466 22191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:37061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=38073
--tserver_master_addrs=127.29.84.190:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:27.618474 22191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:27.619449 22191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:27.629231 22254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:27.629187 22253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:27.629192 22256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:27.629660 22191 server_base.cc:1034] running on GCE node
I20260806 00:47:27.630100 22191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:27.630699 22191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:27.631909 22191 hybrid_clock.cc:648] HybridClock initialized: now 1785977247631869 us; error 43 us; skew 500 ppm
Aug 06 00:47:27 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:27.634747 22191 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:47:27.635692 22191 webserver.cc:460] Webserver started at http://127.29.84.129:38073/ using document root <none> and password file <none>
I20260806 00:47:27.636245 22191 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:27.636334 22191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:27.640039 22191 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 00:47:27.642175 22266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:27.643491 22191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:27.643621 22191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "eb0731dceffc4bb8860ac3e9c39bca0d"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "8e6fb7086d809f59091a675940ca94c4"
server_key_iv: "72e2a4527e01a5c7bec62ef68e02076e"
server_key_version: "encryptionkey@0"
I20260806 00:47:27.644131 22191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:27.672858 22191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:27.675920 22191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:27.676115 22191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:27.676716 22191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:27.677843 22276 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:47:27.683144 22191 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:47:27.683223 22191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 00:47:27.683302 22191 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:47:27.684894 22191 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:47:27.684964 22191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 00:47:27.685282 22276 tablet_bootstrap.cc:492] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Bootstrap starting.
I20260806 00:47:27.691887 22276 log.cc:824] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:27.695835 22191 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:37061
I20260806 00:47:27.696054 22426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:37061 every 8 connection(s)
I20260806 00:47:27.696715 22191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:47:27.697211 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 22191
I20260806 00:47:27.697745 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:46439
--local_ip_for_outbound_sockets=127.29.84.130
--tserver_master_addrs=127.29.84.190:38971
--webserver_port=46149
--webserver_interface=127.29.84.130
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:47:27.701303 22276 tablet_bootstrap.cc:492] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:47:27.701766 22276 tablet_bootstrap.cc:492] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Bootstrap complete.
I20260806 00:47:27.702118 22276 ts_tablet_manager.cc:1397] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.000s
I20260806 00:47:27.705991 22276 raft_consensus.cc:348] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:27.706663 22276 raft_consensus.cc:729] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0731dceffc4bb8860ac3e9c39bca0d, State: Initialized, Role: FOLLOWER
I20260806 00:47:27.707204 22276 consensus_queue.cc:260] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:27.708294 22276 ts_tablet_manager.cc:1428] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260806 00:47:27.765776 22431 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:47:27.699505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:41427)
0806 00:47:27.700044 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:27.700094 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:27.701129 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:27.765030 (+ 63901us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":156,"threads_started":1}
W20260806 00:47:27.766371 22427 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:38971 (0 consecutive failures): Network error: Failed to ping master at 127.29.84.190:38971: Client connection negotiation failed: client connection to 127.29.84.190:38971: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:47:27.767182 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:27.766888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:42433)
0806 00:47:27.767015 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:27.767127 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 00:47:27.767876 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:27.767698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:55841)
0806 00:47:27.767795 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:27.767833 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:47:27.768126 22427 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:47:27.814060 22430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:27.814267 22430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:27.814339 22430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:27.818356 22430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:27.818400 22430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:27.818490 22430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:47:27.823698 22430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:46439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=46149
--tserver_master_addrs=127.29.84.190:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:27.824550 22430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:27.825488 22430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:27.834528 22439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:27.834517 22443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:27.834518 22441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:27.834807 22430 server_base.cc:1034] running on GCE node
I20260806 00:47:27.835211 22430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:27.835772 22430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:27.836963 22430 hybrid_clock.cc:648] HybridClock initialized: now 1785977247836955 us; error 30 us; skew 500 ppm
Aug 06 00:47:27 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:27.839604 22430 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:47:27.840368 22430 webserver.cc:460] Webserver started at http://127.29.84.130:46149/ using document root <none> and password file <none>
I20260806 00:47:27.840886 22430 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:27.840977 22430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:27.844457 22430 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 00:47:27.846315 22453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:27.847601 22430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 00:47:27.847728 22430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "c46ef9b66b994566ac6f137bf9cf0a70"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "5e39a7bca97c3ff312552e2b738976a3"
server_key_iv: "bb3ba89c54407fa4585595b603009327"
server_key_version: "encryptionkey@0"
I20260806 00:47:27.848217 22430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:27.865396 22430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:27.868052 22430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:27.868237 22430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:27.868760 22430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:27.869521 22430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:27.869592 22430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:27.869659 22430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:27.869712 22430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:27.878300 22430 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:46439
I20260806 00:47:27.878372 22617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:46439 every 8 connection(s)
I20260806 00:47:27.879356 22430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:47:27.882814 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:27.881267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:34931)
0806 00:47:27.881741 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:27.882147 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
W20260806 00:47:27.883253 22618 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:38971 (0 consecutive failures): Network error: Failed to ping master at 127.29.84.190:38971: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
I20260806 00:47:27.883826 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:27.883610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:46887)
0806 00:47:27.883724 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:27.883776 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 00:47:27.884404 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:27.884204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:60075)
0806 00:47:27.884305 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:27.884351 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 00:47:27.884605 22618 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:47:27.887907 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 22430
I20260806 00:47:27.888326 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:34323
--local_ip_for_outbound_sockets=127.29.84.131
--tserver_master_addrs=127.29.84.190:38971
--webserver_port=40469
--webserver_interface=127.29.84.131
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:27.992964 22622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:27.993172 22622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:27.993211 22622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:27.997117 22622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:27.997164 22622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:27.997241 22622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:47:28.002653 22622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:34323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=40469
--tserver_master_addrs=127.29.84.190:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:28.003566 22622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:28.004487 22622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:28.013180 22628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:28.013173 22632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:28.013173 22629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:28.013510 22622 server_base.cc:1034] running on GCE node
I20260806 00:47:28.013875 22622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:28.014426 22622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:28.015630 22622 hybrid_clock.cc:648] HybridClock initialized: now 1785977248015605 us; error 42 us; skew 500 ppm
Aug 06 00:47:28 dist-test-slave-n326 krb5kdc[20550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977248, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:28.018239 22622 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:47:28.018925 22622 webserver.cc:460] Webserver started at http://127.29.84.131:40469/ using document root <none> and password file <none>
I20260806 00:47:28.019454 22622 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:28.019515 22622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:28.022912 22622 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:28.024797 22640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:28.025883 22622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:47:28.025974 22622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "26700deaed384233a392d117c0ce3225"
format_stamp: "Formatted at 2026-08-06 00:46:45 on dist-test-slave-n326"
server_key: "9c24261d2d0aec00b6df0605bda2d80f"
server_key_iv: "76cbaf7bbc32abb8f1e30662e8eaaa33"
server_key_version: "encryptionkey@0"
I20260806 00:47:28.026433 22622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:28.038003 22622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:28.040922 22622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:28.041064 22622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:28.041527 22622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:28.042217 22622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:28.042263 22622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:28.042299 22622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:28.042321 22622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:28.050951 22622 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:34323
I20260806 00:47:28.051033 22801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:34323 every 8 connection(s)
I20260806 00:47:28.051856 22622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:47:28.055207 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:28.053707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:59581)
0806 00:47:28.054100 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:28.054494 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":95,"threads_started":1}
I20260806 00:47:28.055387 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 22622
W20260806 00:47:28.055644 22803 heartbeater.cc:644] Failed to heartbeat to 127.29.84.190:38971 (0 consecutive failures): Network error: Failed to ping master at 127.29.84.190:38971: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
I20260806 00:47:28.056201 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:28.055956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:47069)
0806 00:47:28.056082 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:28.056138 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 00:47:28.056843 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:28.056666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:36677)
0806 00:47:28.056759 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:28.056800 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 00:47:28.057037 22803 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:47:28.769095 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:28.768686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:42943)
0806 00:47:28.768942 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:28.769002 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
I20260806 00:47:28.885458 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:28.885084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:56607)
0806 00:47:28.885316 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:28.885366 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":67,"threads_started":1}
I20260806 00:47:29.057938 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:29.057511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:52865)
0806 00:47:29.057798 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:29.057849 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":63,"threads_started":1}
I20260806 00:47:29.328497 22812 raft_consensus.cc:482] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:29.328751 22812 raft_consensus.cc:504] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:29.329967 22812 leader_election.cc:288] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439), 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323)
Aug 06 00:47:29 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.130@KRBTEST.COM
Aug 06 00:47:29 dist-test-slave-n326 krb5kdc[20550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.131@KRBTEST.COM
I20260806 00:47:29.344511 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:29.330335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:35217 (local address 127.29.84.130:46439)
0806 00:47:29.330784 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:29.330812 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:29.330860 (+ 48us) server_negotiation.cc:407] Connection header received
0806 00:47:29.331613 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:29.331618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:29.331956 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:29.332192 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:29.333496 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:29.334425 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:29.335827 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:29.336334 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:29.336478 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:29.339227 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:29.339232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:29.339243 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:29.339268 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:29.341319 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:29.341964 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:29.341970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:29.341974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:29.342153 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:29.342594 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:29.342597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:29.342599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:29.343137 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:29.343273 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:29.344046 (+ 773us) server_negotiation.cc:299] Negotiation successful
0806 00:47:29.344319 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":86,"threads_started":1}
I20260806 00:47:29.344537 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:29.330138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:46439 (local address 127.29.84.129:35217)
0806 00:47:29.330425 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:29.330442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:29.330591 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:29.332307 (+ 1716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:29.332313 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:29.332856 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:29.333301 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:29.333316 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:29.334591 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:29.334599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:29.335674 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:29.335686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:29.336639 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:29.336641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:29.336712 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:29.337315 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:29.337330 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:29.338994 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:29.341457 (+ 2463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:29.341462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:29.341473 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:29.341851 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:29.342270 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:29.342274 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:29.342277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:29.342488 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:29.343261 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:29.343267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:29.343688 (+ 421us) client_negotiation.cc:769] Sending connection context
0806 00:47:29.343903 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 00:47:29.344288 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":123,"spinlock_wait_cycles":20224,"thread_start_us":91,"threads_started":1}
I20260806 00:47:29.344538 22815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:29.330534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:34323 (local address 127.29.84.129:49137)
0806 00:47:29.330835 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:29.330849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:29.330948 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:29.332305 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:29.332312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:29.332833 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:29.333302 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:29.333315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:29.334600 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:29.334602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:29.335657 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:29.335668 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:29.336465 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:29.336469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:29.336650 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:29.337361 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:29.337373 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:29.339381 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:29.341653 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:29.341658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:29.341663 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:29.341959 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:29.342288 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:29.342290 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:29.342292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:29.342488 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:29.343261 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:29.343267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:29.343688 (+ 421us) client_negotiation.cc:769] Sending connection context
0806 00:47:29.343903 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 00:47:29.344342 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":424,"spinlock_wait_cycles":19328,"thread_start_us":63,"threads_started":1}
I20260806 00:47:29.344533 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:29.330704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:49137 (local address 127.29.84.131:34323)
0806 00:47:29.330998 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:29.331029 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:29.331057 (+ 28us) server_negotiation.cc:407] Connection header received
0806 00:47:29.331732 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:29.331737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:29.332028 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:29.332317 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:29.333485 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:29.334431 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:29.335788 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:29.336334 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:29.336506 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:29.339587 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:29.339594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:29.339612 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:29.339641 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:29.341549 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:29.342065 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:29.342070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:29.342075 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:29.342200 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:29.342594 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:29.342597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:29.342599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:29.343137 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:29.343273 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:29.344040 (+ 767us) server_negotiation.cc:299] Negotiation successful
0806 00:47:29.344319 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":65,"threads_started":1}
W20260806 00:47:29.347559 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
W20260806 00:47:29.347790 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
I20260806 00:47:29.347882 22269 leader_election.cc:302] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0731dceffc4bb8860ac3e9c39bca0d; no voters: 26700deaed384233a392d117c0ce3225, c46ef9b66b994566ac6f137bf9cf0a70
I20260806 00:47:29.348124 22812 raft_consensus.cc:2726] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:47:29.770044 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:29.769763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:38903)
0806 00:47:29.769909 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:29.769963 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 00:47:29.886498 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:29.886112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:33007)
0806 00:47:29.886348 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:29.886417 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":63,"threads_started":1}
I20260806 00:47:30.058959 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:30.058614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:59225)
0806 00:47:30.058831 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:30.058881 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260806 00:47:30.771211 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:30.770758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:49055)
0806 00:47:30.771011 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:30.771110 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":76,"threads_started":1}
I20260806 00:47:30.887590 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:30.887203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:42731)
0806 00:47:30.887442 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:30.887494 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":68,"threads_started":1}
I20260806 00:47:31.059998 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:31.059613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:36253)
0806 00:47:31.059856 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:31.059903 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":92,"threads_started":1}
I20260806 00:47:31.370312 22823 raft_consensus.cc:482] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:31.370445 22823 raft_consensus.cc:504] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:31.370951 22823 leader_election.cc:288] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439), 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323)
W20260806 00:47:31.371831 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
W20260806 00:47:31.372000 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
I20260806 00:47:31.372068 22269 leader_election.cc:302] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0731dceffc4bb8860ac3e9c39bca0d; no voters: 26700deaed384233a392d117c0ce3225, c46ef9b66b994566ac6f137bf9cf0a70
I20260806 00:47:31.372255 22823 raft_consensus.cc:2726] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:47:31.772326 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:31.771954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:56769)
0806 00:47:31.772180 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:31.772232 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":71,"threads_started":1}
I20260806 00:47:31.888700 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:31.888307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:33801)
0806 00:47:31.888569 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:31.888619 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":117,"threads_started":1}
I20260806 00:47:32.061048 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:32.060606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:32903)
0806 00:47:32.060921 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:32.060972 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":64,"threads_started":1}
I20260806 00:47:32.773772 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:32.773159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:35115)
0806 00:47:32.773501 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:32.773561 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":74,"threads_started":1}
I20260806 00:47:32.889822 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:32.889445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:47923)
0806 00:47:32.889686 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:32.889738 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":71,"threads_started":1}
I20260806 00:47:33.062150 22831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:33.061757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:55763)
0806 00:47:33.062015 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:33.062068 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":84,"threads_started":1}
I20260806 00:47:33.695048 22832 raft_consensus.cc:482] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:33.695293 22832 raft_consensus.cc:504] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:33.695732 22832 leader_election.cc:288] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439), 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323)
W20260806 00:47:33.696663 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
W20260806 00:47:33.696841 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
I20260806 00:47:33.696903 22269 leader_election.cc:302] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0731dceffc4bb8860ac3e9c39bca0d; no voters: 26700deaed384233a392d117c0ce3225, c46ef9b66b994566ac6f137bf9cf0a70
I20260806 00:47:33.697052 22832 raft_consensus.cc:2726] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:47:33.774863 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:33.774479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:55917)
0806 00:47:33.774720 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:33.774772 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":80,"threads_started":1}
I20260806 00:47:33.890988 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:33.890589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:58011)
0806 00:47:33.890855 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:33.890908 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":86,"threads_started":1}
I20260806 00:47:34.063273 22836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:34.062863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:59017)
0806 00:47:34.063148 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:34.063196 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":69,"threads_started":1}
I20260806 00:47:34.776017 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:34.775590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:56273)
0806 00:47:34.775864 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:34.775919 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":96,"threads_started":1}
I20260806 00:47:34.892079 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:34.891653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:42229)
0806 00:47:34.891928 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:34.891981 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":69,"threads_started":1}
I20260806 00:47:35.064278 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:35.063884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:51865)
0806 00:47:35.064151 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:35.064201 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":80,"threads_started":1}
I20260806 00:47:35.777330 22844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:35.776914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:49603)
0806 00:47:35.777185 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:35.777237 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":84,"threads_started":1}
I20260806 00:47:35.893098 22845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:35.892734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:34443)
0806 00:47:35.892958 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:35.893018 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260806 00:47:36.065341 22846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:36.064986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:44479)
0806 00:47:36.065207 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:36.065255 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260806 00:47:36.259744 22847 raft_consensus.cc:482] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:36.259881 22847 raft_consensus.cc:504] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:36.260326 22847 leader_election.cc:288] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439), 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323)
W20260806 00:47:36.261145 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
W20260806 00:47:36.261271 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
I20260806 00:47:36.261309 22269 leader_election.cc:302] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0731dceffc4bb8860ac3e9c39bca0d; no voters: 26700deaed384233a392d117c0ce3225, c46ef9b66b994566ac6f137bf9cf0a70
I20260806 00:47:36.261454 22847 raft_consensus.cc:2726] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:47:36.778393 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:36.778009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:57007)
0806 00:47:36.778251 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:36.778299 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260806 00:47:36.894080 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:36.893713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:60865)
0806 00:47:36.893940 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:36.893991 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":72,"threads_started":1}
I20260806 00:47:37.066444 22851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:37.066074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:34541)
0806 00:47:37.066312 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:37.066364 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":68,"threads_started":1}
I20260806 00:47:37.779680 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:37.779272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:49873)
0806 00:47:37.779517 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:37.779574 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":70,"threads_started":1}
I20260806 00:47:37.895195 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:37.894754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:57299)
0806 00:47:37.895001 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:37.895096 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
I20260806 00:47:38.067591 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:38.067235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:33689)
0806 00:47:38.067462 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:38.067514 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
I20260806 00:47:38.085649 22857 raft_consensus.cc:482] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:38.085809 22857 raft_consensus.cc:504] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46ef9b66b994566ac6f137bf9cf0a70" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 46439 } } peers { permanent_uuid: "26700deaed384233a392d117c0ce3225" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34323 } } peers { permanent_uuid: "eb0731dceffc4bb8860ac3e9c39bca0d" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 37061 } }
I20260806 00:47:38.086226 22857 leader_election.cc:288] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439), 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323)
W20260806 00:47:38.087035 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c46ef9b66b994566ac6f137bf9cf0a70 (127.29.84.130:46439): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
W20260806 00:47:38.087242 22269 leader_election.cc:341] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26700deaed384233a392d117c0ce3225 (127.29.84.131:34323): Not found: Tablet not found: 7856871feea140b8b1bd0e1c5a837800
I20260806 00:47:38.087306 22269 leader_election.cc:302] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0731dceffc4bb8860ac3e9c39bca0d; no voters: 26700deaed384233a392d117c0ce3225, c46ef9b66b994566ac6f137bf9cf0a70
I20260806 00:47:38.087512 22857 raft_consensus.cc:2726] T 7856871feea140b8b1bd0e1c5a837800 P eb0731dceffc4bb8860ac3e9c39bca0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:47:38.780823 22859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:38.780416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:52929)
0806 00:47:38.780678 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:38.780729 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":90,"threads_started":1}
I20260806 00:47:38.896252 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:38.895835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:41135)
0806 00:47:38.896119 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:38.896169 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":101,"threads_started":1}
I20260806 00:47:39.068598 22861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:39.068241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:45849)
0806 00:47:39.068467 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:39.068519 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":69,"threads_started":1}
I20260806 00:47:39.782045 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:39.781636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:57077)
0806 00:47:39.781888 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:39.781951 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":71,"threads_started":1}
I20260806 00:47:39.897364 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:39.896983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:34005)
0806 00:47:39.897228 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:39.897281 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260806 00:47:40.069635 22869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:40.069244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:46679)
0806 00:47:40.069487 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:40.069555 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":69,"threads_started":1}
I20260806 00:47:40.783263 22870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:40.782760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:32963)
0806 00:47:40.783036 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:40.783149 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":72,"threads_started":1}
I20260806 00:47:40.898473 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:40.898050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:55931)
0806 00:47:40.898334 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:40.898387 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":127,"threads_started":1}
I20260806 00:47:41.070690 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:41.070300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:55349)
0806 00:47:41.070552 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:41.070607 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260806 00:47:41.784417 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:41.783992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:56187)
0806 00:47:41.784270 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:41.784322 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":102,"threads_started":1}
I20260806 00:47:41.899690 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:41.899250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:59083)
0806 00:47:41.899541 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:41.899595 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":110,"threads_started":1}
I20260806 00:47:42.071797 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:42.071417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:36693)
0806 00:47:42.071654 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:42.071707 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
I20260806 00:47:42.785882 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:42.785354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.129:32875)
0806 00:47:42.785682 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:42.785755 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":116,"threads_started":1}
I20260806 00:47:42.900900 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:42.900555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.130:59543)
0806 00:47:42.900773 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:42.900821 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":71,"threads_started":1}
I20260806 00:47:43.061998 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 22191
I20260806 00:47:43.072983 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:43.072614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38971 (local address 127.29.84.131:43109)
0806 00:47:43.072853 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:43.072903 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.190:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260806 00:47:43.073623 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 22430
I20260806 00:47:43.083560 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 22622
2026-08-06T00:47:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59290 ms)
[----------] 30 tests from SecurityITest (242252 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:47:43 dist-test-slave-n326 krb5kdc[22886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:47:43 dist-test-slave-n326 krb5kdc[22886](info): set up 2 sockets
Aug 06 00:47:43 dist-test-slave-n326 krb5kdc[22886](info): commencing operation
krb5kdc: starting...
W20260806 00:47:44.313442 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:47:44 dist-test-slave-n326 krb5kdc[22886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:47:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:47:44.434478 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40683
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:40683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:44.540977 22910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:44.541185 22910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:44.541222 22910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:44.545012 22910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:44.545058 22910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:44.545075 22910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:44.545091 22910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:44.545105 22910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:44.549923 22910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:40683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:44.550695 22910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:44.551575 22910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:44.559274 22919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:44.559306 22916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:44.559315 22917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:44.559347 22910 server_base.cc:1034] running on GCE node
I20260806 00:47:44.559953 22910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:44.560922 22910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:44.562095 22910 hybrid_clock.cc:648] HybridClock initialized: now 1785977264562073 us; error 35 us; skew 500 ppm
Aug 06 00:47:44 dist-test-slave-n326 krb5kdc[22886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977264, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:44.564783 22910 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:47:44.565589 22910 webserver.cc:460] Webserver started at http://127.29.84.190:45413/ using document root <none> and password file <none>
I20260806 00:47:44.566079 22910 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:44.566162 22910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:44.566380 22910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:44.568243 22910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b"
format_stamp: "Formatted at 2026-08-06 00:47:44 on dist-test-slave-n326"
server_key: "f1ecc47002f6df31ceab93155c2a616c"
server_key_iv: "6a4a3a2c8a3ac23deba05334e65310dc"
server_key_version: "encryptionkey@0"
I20260806 00:47:44.568750 22910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b"
format_stamp: "Formatted at 2026-08-06 00:47:44 on dist-test-slave-n326"
server_key: "f1ecc47002f6df31ceab93155c2a616c"
server_key_iv: "6a4a3a2c8a3ac23deba05334e65310dc"
server_key_version: "encryptionkey@0"
I20260806 00:47:44.571939 22910 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:47:44.574020 22928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:44.575124 22910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:44.575241 22910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b"
format_stamp: "Formatted at 2026-08-06 00:47:44 on dist-test-slave-n326"
server_key: "f1ecc47002f6df31ceab93155c2a616c"
server_key_iv: "6a4a3a2c8a3ac23deba05334e65310dc"
server_key_version: "encryptionkey@0"
I20260806 00:47:44.575338 22910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:44.590516 22910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:44.593415 22910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:44.593602 22910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:44.600755 22910 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:40683
I20260806 00:47:44.600769 23005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:40683 every 8 connection(s)
I20260806 00:47:44.601636 22910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:44.604142 23006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:44.609526 23006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b: Bootstrap starting.
I20260806 00:47:44.609972 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 22910
I20260806 00:47:44.610107 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:44.610445 30034 external_mini_cluster.cc:1428] Setting key dbc6ee5a28dcf51be481b93f76004b46
I20260806 00:47:44.612264 23006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:44.613150 23006 log.cc:824] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:44.614988 23006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b: No bootstrap required, opened a new log
I20260806 00:47:44.618583 23006 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40683 } }
I20260806 00:47:44.618852 23006 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:44.618901 23006 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf0a3177ac843b3ba14bbe7bdb03c1b, State: Initialized, Role: FOLLOWER
Aug 06 00:47:44 dist-test-slave-n326 krb5kdc[22886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:44.619344 23006 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40683 } }
I20260806 00:47:44.619498 23006 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:44.619596 23006 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:44.619702 23006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:44.620889 23006 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40683 } }
I20260806 00:47:44.621234 23006 leader_election.cc:302] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdf0a3177ac843b3ba14bbe7bdb03c1b; no voters:
I20260806 00:47:44.621582 23006 leader_election.cc:288] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:47:44.621671 23012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:44.621927 23012 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [term 1 LEADER]: Becoming Leader. State: Replica: bdf0a3177ac843b3ba14bbe7bdb03c1b, State: Running, Role: LEADER
I20260806 00:47:44.622257 23012 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40683 } }
I20260806 00:47:44.622936 23006 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:44.623656 23015 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40683 } } }
I20260806 00:47:44.623805 23015 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:44.624207 23023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:44.624254 23016 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdf0a3177ac843b3ba14bbe7bdb03c1b. Latest consensus state: current_term: 1 leader_uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0a3177ac843b3ba14bbe7bdb03c1b" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40683 } } }
I20260806 00:47:44.624404 23016 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:44.625331 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:44.611740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.29.84.190:40683)
0806 00:47:44.612195 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:44.612209 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:44.612241 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:47:44.612876 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:44.612891 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:44.613190 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:44.613496 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:44.614284 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:44.615321 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:44.616169 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:44.616628 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:44.616790 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:44.619624 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:44.619630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:44.619646 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:44.619676 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:44.621643 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:44.622189 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:44.622195 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:44.622200 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:44.622325 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:44.622646 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:44.622649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:44.622651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:44.623148 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:44.623923 (+ 775us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:44.624075 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 00:47:44.624358 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":137,"threads_started":1}
I20260806 00:47:44.627228 23023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:44.632047 23023 catalog_manager.cc:1347] Generated new cluster ID: ccfb1317675c444d97f51bd8ffd0d1e6
I20260806 00:47:44.632107 23023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:44.646134 23023 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:47:44.646952 23023 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:44.652546 23023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b: Generated new TSK 0
I20260806 00:47:44.653076 23023 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:47:44.688302 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40683
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:44.796871 23042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:44.797117 23042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:44.797189 23042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:44.801404 23042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:44.801514 23042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:44.801596 23042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:47:44.806633 23042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:44.807631 23042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:44.808624 23042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:47:44.817791 23042 server_base.cc:1034] running on GCE node
W20260806 00:47:44.817693 23050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:44.817710 23056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:44.817726 23053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:44.818305 23042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:44.818832 23042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:44.819988 23042 hybrid_clock.cc:648] HybridClock initialized: now 1785977264819968 us; error 31 us; skew 500 ppm
Aug 06 00:47:44 dist-test-slave-n326 krb5kdc[22886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977264, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:44.823032 23042 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:47:44.823974 23042 webserver.cc:460] Webserver started at http://127.29.84.129:42893/ using document root <none> and password file <none>
I20260806 00:47:44.824623 23042 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:44.824710 23042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:44.824903 23042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:44.826754 23042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "089726e841e944828f84758edd5b33ef"
format_stamp: "Formatted at 2026-08-06 00:47:44 on dist-test-slave-n326"
server_key: "72e44d1d06f75f1589a6d7160614dcf3"
server_key_iv: "5efd385156d543bcaa449b85994cd637"
server_key_version: "encryptionkey@0"
I20260806 00:47:44.827309 23042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "089726e841e944828f84758edd5b33ef"
format_stamp: "Formatted at 2026-08-06 00:47:44 on dist-test-slave-n326"
server_key: "72e44d1d06f75f1589a6d7160614dcf3"
server_key_iv: "5efd385156d543bcaa449b85994cd637"
server_key_version: "encryptionkey@0"
I20260806 00:47:44.830684 23042 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:47:44.832918 23067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:44.834069 23042 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:44.834187 23042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "089726e841e944828f84758edd5b33ef"
format_stamp: "Formatted at 2026-08-06 00:47:44 on dist-test-slave-n326"
server_key: "72e44d1d06f75f1589a6d7160614dcf3"
server_key_iv: "5efd385156d543bcaa449b85994cd637"
server_key_version: "encryptionkey@0"
I20260806 00:47:44.834285 23042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:44.844135 23042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:44.846859 23042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:44.847007 23042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:44.847535 23042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:44.848219 23042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:44.848264 23042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:44.848299 23042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:44.848318 23042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:44.857092 23042 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:35653
I20260806 00:47:44.857120 23231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:35653 every 8 connection(s)
I20260806 00:47:44.857860 23042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:47:44.864060 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 23042
I20260806 00:47:44.864204 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:47:44.864473 30034 external_mini_cluster.cc:1428] Setting key 58ce67372cdd753fa38cfd3c2c3ef6d9
Aug 06 00:47:44 dist-test-slave-n326 krb5kdc[22886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977264, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:44.871933 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:44.859497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:41805 (local address 127.29.84.190:40683)
0806 00:47:44.859675 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:44.859680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:44.860917 (+ 1237us) server_negotiation.cc:407] Connection header received
0806 00:47:44.862089 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:44.862092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:44.862137 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:44.862208 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:44.863305 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:44.863849 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:44.865196 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:44.865457 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:44.865590 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:44.868190 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:44.868194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:44.868197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:44.868224 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:44.869642 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:44.870270 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:44.870272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:44.870274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:44.870348 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:44.870774 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:44.870777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:44.870778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:44.870944 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:44.871022 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:44.871705 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 00:47:44.871808 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":69}
I20260806 00:47:44.873044 23234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:44.859946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40683 (local address 127.29.84.129:41805)
0806 00:47:44.860793 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:44.860831 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:44.861921 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:44.862335 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:44.862341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:44.862662 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:44.863146 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:44.863168 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:44.864002 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:44.864006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:44.865091 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:44.865105 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:44.865651 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:44.865655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:44.865841 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:44.866386 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:44.866402 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:44.868004 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:44.869757 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:44.869765 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:44.869780 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:44.870149 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:44.870469 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:44.870475 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:44.870480 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:44.870671 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:44.871091 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:44.871098 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:44.871418 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 00:47:44.871636 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 00:47:44.871995 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":157,"threads_started":1}
I20260806 00:47:44.874205 23232 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40683
I20260806 00:47:44.874426 23232 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:44.874846 23232 heartbeater.cc:511] Master 127.29.84.190:40683 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
I20260806 00:47:44.876327 22961 ts_manager.cc:194] Registered new tserver with Master: 089726e841e944828f84758edd5b33ef (127.29.84.129:35653)
I20260806 00:47:44.877425 22961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:41805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:47:44.893734 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40683
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:45.002276 23238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:45.002502 23238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:45.002555 23238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:45.006337 23238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:45.006387 23238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:45.006471 23238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:47:45.011731 23238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:45.012571 23238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:45.013442 23238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:45.021965 23254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:45.021974 23251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:45.021965 23250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:45.022022 23238 server_base.cc:1034] running on GCE node
I20260806 00:47:45.022634 23238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:45.023231 23238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:45.024385 23238 hybrid_clock.cc:648] HybridClock initialized: now 1785977265024378 us; error 32 us; skew 500 ppm
Aug 06 00:47:45 dist-test-slave-n326 krb5kdc[22886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977265, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:45.026888 23238 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:47:45.027688 23238 webserver.cc:460] Webserver started at http://127.29.84.130:32771/ using document root <none> and password file <none>
I20260806 00:47:45.028195 23238 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:45.028301 23238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:45.028553 23238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:45.030364 23238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "897b9a79dd094ed081dd24b4ab0d0162"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "33058a5f433b12589fa0d18a5847c418"
server_key_iv: "c5e7383d128b8a9298621929dbf363aa"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.030846 23238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "897b9a79dd094ed081dd24b4ab0d0162"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "33058a5f433b12589fa0d18a5847c418"
server_key_iv: "c5e7383d128b8a9298621929dbf363aa"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.034174 23238 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:47:45.036476 23262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.037629 23238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:47:45.037719 23238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "897b9a79dd094ed081dd24b4ab0d0162"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "33058a5f433b12589fa0d18a5847c418"
server_key_iv: "c5e7383d128b8a9298621929dbf363aa"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.037869 23238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:45.062093 23238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:45.064570 23238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:45.064702 23238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:45.065151 23238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:45.065822 23238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:45.065864 23238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.065896 23238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:45.065922 23238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.073976 23238 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:45165
I20260806 00:47:45.074047 23418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:45165 every 8 connection(s)
I20260806 00:47:45.074741 23238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:47:45.079335 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 23238
I20260806 00:47:45.079499 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:47:45.079829 30034 external_mini_cluster.cc:1428] Setting key 192fa07569113872b58afba0726dee32
Aug 06 00:47:45 dist-test-slave-n326 krb5kdc[22886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977265, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:45.088281 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.076329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:47631 (local address 127.29.84.190:40683)
0806 00:47:45.076491 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.076496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.077665 (+ 1169us) server_negotiation.cc:407] Connection header received
0806 00:47:45.078436 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.078439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.078510 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.078611 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:45.079923 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.080616 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.082027 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.082239 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.082315 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.084842 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:45.084846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:45.084854 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:45.084876 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:45.086308 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:45.086797 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:45.086801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:45.086803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:45.086880 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:45.087275 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:45.087280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:45.087282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:45.087452 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:45.087555 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.088003 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.088116 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:47:45.088954 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.076690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40683 (local address 127.29.84.130:47631)
0806 00:47:45.077529 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.077568 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:45.078257 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:45.078749 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:45.078755 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:45.079044 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:45.079760 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.079781 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.080730 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.080732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.081891 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.081905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.082380 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.082387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.082613 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.083171 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:45.083196 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:45.084690 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:45.086432 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:45.086438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:45.086451 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:45.086695 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:45.086978 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:45.086982 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:45.086984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:45.087177 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:45.087554 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:45.087560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:45.087777 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 00:47:45.087945 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 00:47:45.088229 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":539,"threads_started":1}
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:47:45.090144 23420 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40683
I20260806 00:47:45.090394 23420 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:45.090822 23420 heartbeater.cc:511] Master 127.29.84.190:40683 requested a full tablet report, sending...
I20260806 00:47:45.091807 22961 ts_manager.cc:194] Registered new tserver with Master: 897b9a79dd094ed081dd24b4ab0d0162 (127.29.84.130:45165)
I20260806 00:47:45.092401 22961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:47631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:47:45.107259 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40683
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:45.215448 23426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:45.215658 23426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:45.215715 23426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:45.219851 23426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:45.219900 23426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:45.219954 23426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:47:45.225088 23426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:45.225919 23426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:45.226797 23426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:45.235842 23436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:45.235831 23433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:45.235884 23439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:45.236058 23426 server_base.cc:1034] running on GCE node
I20260806 00:47:45.236497 23426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:45.237107 23426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:45.238279 23426 hybrid_clock.cc:648] HybridClock initialized: now 1785977265238242 us; error 58 us; skew 500 ppm
Aug 06 00:47:45 dist-test-slave-n326 krb5kdc[22886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977265, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:45.240952 23426 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:47:45.241817 23426 webserver.cc:460] Webserver started at http://127.29.84.131:35181/ using document root <none> and password file <none>
I20260806 00:47:45.242290 23426 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:45.242348 23426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:45.242498 23426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:45.244391 23426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "bca56b4f5c3fbe0f9348cec31f87b2ef"
server_key_iv: "96c25ee83f8289e37331d66b44bfd84e"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.244864 23426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "bca56b4f5c3fbe0f9348cec31f87b2ef"
server_key_iv: "96c25ee83f8289e37331d66b44bfd84e"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.248242 23426 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:47:45.250351 23447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.251487 23426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:45.251592 23426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "bca56b4f5c3fbe0f9348cec31f87b2ef"
server_key_iv: "96c25ee83f8289e37331d66b44bfd84e"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.251667 23426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:45.265681 23426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:45.268200 23426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:45.268332 23426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:45.268813 23426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:45.269512 23426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:45.269572 23426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.269608 23426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:45.269621 23426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.279860 23426 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:39361
I20260806 00:47:45.279883 23602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:39361 every 8 connection(s)
I20260806 00:47:45.280644 23426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:47:45.282923 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 23426
I20260806 00:47:45.283092 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:47:45.283406 30034 external_mini_cluster.cc:1428] Setting key 968f416576159425b962e4e935ad98c5
Aug 06 00:47:45 dist-test-slave-n326 krb5kdc[22886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977265, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:45.293167 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.282261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:42147 (local address 127.29.84.190:40683)
0806 00:47:45.282419 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.282424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.283230 (+ 806us) server_negotiation.cc:407] Connection header received
0806 00:47:45.284481 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.284485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.284552 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.284645 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:45.285701 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.286223 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.287399 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.287626 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.287731 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.289878 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:45.289882 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:45.289889 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:45.289913 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:45.291246 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:45.291726 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:45.291730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:45.291732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:45.291815 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:45.292146 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:45.292150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:45.292152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:45.292327 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:45.292428 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.292884 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.292999 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:47:45.293895 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.282631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40683 (local address 127.29.84.131:42147)
0806 00:47:45.283093 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.283142 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:45.284276 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:45.284815 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:45.284821 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:45.285131 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:45.285567 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.285579 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.286345 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.286349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.287280 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.287290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.287779 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.287784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.287944 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.288410 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:45.288425 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:45.289732 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:45.291358 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:45.291364 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:45.291376 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:45.291622 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:45.291907 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:45.291911 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:45.291913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:45.292049 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:45.292442 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:45.292450 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:45.292665 (+ 215us) client_negotiation.cc:769] Sending connection context
0806 00:47:45.292838 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 00:47:45.293135 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":127,"threads_started":1}
I20260806 00:47:45.294934 23606 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40683
I20260806 00:47:45.295190 23606 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:45.295655 23606 heartbeater.cc:511] Master 127.29.84.190:40683 requested a full tablet report, sending...
I20260806 00:47:45.296705 22960 ts_manager.cc:194] Registered new tserver with Master: 2bb1dfb4cde943e6ba3d4b251a3b2b8e (127.29.84.131:39361)
I20260806 00:47:45.297263 22960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:42147
I20260806 00:47:45.308492 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:47:45.317525 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.310606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59232 (local address 127.29.84.190:40683)
0806 00:47:45.310743 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.310748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.310951 (+ 203us) server_negotiation.cc:407] Connection header received
0806 00:47:45.311050 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.311052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.311128 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.311193 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:45.311978 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.312448 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.313256 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.313454 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.313531 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.314693 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:45.314696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:45.314697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:45.314717 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:45.316088 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:45.316460 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:45.316462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:45.316463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:45.316529 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:45.316825 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:45.316827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:45.316828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:45.317014 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:45.317111 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.317277 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.317360 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:47:45.320188 22960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:47:45.323050 22960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:47:45.337216 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.331832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:35653 (local address 127.0.0.1:38568)
0806 00:47:45.332602 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.332616 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:45.332705 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:45.333128 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:45.333133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:45.333154 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:45.333272 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.333280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.334762 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.334765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.335908 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.335917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.336703 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.336707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.336809 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.336843 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 00:47:45.336967 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 00:47:45.337079 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":45,"thread_start_us":90,"threads_started":1}
I20260806 00:47:45.337226 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.331319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.29.84.129:35653)
0806 00:47:45.331697 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.331710 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.332740 (+ 1030us) server_negotiation.cc:407] Connection header received
0806 00:47:45.332825 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.332829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.332981 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.333137 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:45.333409 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.334598 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.336025 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.336582 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.336706 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.336852 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.337002 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.337102 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":162,"threads_started":1}
I20260806 00:47:45.337448 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.331424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:45165 (local address 127.0.0.1:47324)
0806 00:47:45.332159 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.332206 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:45.332315 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:45.333289 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:45.333292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:45.333311 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:45.333405 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.333411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.334530 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.334533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.335960 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.335971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.337079 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.337083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.337186 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.337208 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:47:45.337253 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 00:47:45.337316 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":109,"threads_started":1}
I20260806 00:47:45.338119 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.331564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.29.84.130:45165)
0806 00:47:45.332941 (+ 1377us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.332950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.332973 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:47:45.333032 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.333035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.333164 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.333311 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:45.333524 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.334410 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.336101 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.336942 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.337677 (+ 735us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.337828 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.337879 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.337972 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1285,"thread_start_us":142,"threads_started":1}
I20260806 00:47:45.338986 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.332378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:39361 (local address 127.0.0.1:52446)
0806 00:47:45.333363 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.333377 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:45.333479 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:45.333959 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:45.333962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:45.333982 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:45.334096 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.334102 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.335463 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.335467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.336615 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.336626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.338569 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.338572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.338673 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.338693 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:47:45.338752 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 00:47:45.338818 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":3,"thread_start_us":108,"threads_started":1}
I20260806 00:47:45.339380 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.332521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.29.84.131:39361)
0806 00:47:45.332805 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.332818 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.333572 (+ 754us) server_negotiation.cc:407] Connection header received
0806 00:47:45.333647 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.333652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.333816 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.333965 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:45.334209 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.335311 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.336783 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.337622 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.337696 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.339156 (+ 1460us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.339215 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.339280 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":93,"threads_started":1}
I20260806 00:47:45.340649 23140 tablet_service.cc:1467] Processing CreateTablet for tablet 04181b8d57cd4fa8af511d5ace231a6e (DEFAULT_TABLE table=test-table [id=965abf157f2d4c53829585c7b80ab1bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:45.340651 23326 tablet_service.cc:1467] Processing CreateTablet for tablet 04181b8d57cd4fa8af511d5ace231a6e (DEFAULT_TABLE table=test-table [id=965abf157f2d4c53829585c7b80ab1bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:45.341769 23140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04181b8d57cd4fa8af511d5ace231a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:45.341769 23326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04181b8d57cd4fa8af511d5ace231a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:45.345234 23515 tablet_service.cc:1467] Processing CreateTablet for tablet 04181b8d57cd4fa8af511d5ace231a6e (DEFAULT_TABLE table=test-table [id=965abf157f2d4c53829585c7b80ab1bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:45.346136 23515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04181b8d57cd4fa8af511d5ace231a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:45.349056 23629 tablet_bootstrap.cc:492] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef: Bootstrap starting.
I20260806 00:47:45.349848 23630 tablet_bootstrap.cc:492] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162: Bootstrap starting.
I20260806 00:47:45.351739 23630 tablet_bootstrap.cc:654] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:45.352150 23629 tablet_bootstrap.cc:654] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:45.353045 23630 log.cc:824] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:45.353379 23632 tablet_bootstrap.cc:492] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: Bootstrap starting.
I20260806 00:47:45.353606 23629 log.cc:824] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:45.355348 23632 tablet_bootstrap.cc:654] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:45.355684 23629 tablet_bootstrap.cc:492] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef: No bootstrap required, opened a new log
I20260806 00:47:45.355684 23630 tablet_bootstrap.cc:492] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162: No bootstrap required, opened a new log
I20260806 00:47:45.355880 23629 ts_tablet_manager.cc:1397] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:47:45.355892 23630 ts_tablet_manager.cc:1397] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:47:45.356423 23632 log.cc:824] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:45.358099 23632 tablet_bootstrap.cc:492] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: No bootstrap required, opened a new log
I20260806 00:47:45.358310 23632 ts_tablet_manager.cc:1397] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:47:45.359920 23630 raft_consensus.cc:348] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } }
I20260806 00:47:45.359920 23629 raft_consensus.cc:348] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } }
I20260806 00:47:45.360195 23629 raft_consensus.cc:374] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:45.360196 23630 raft_consensus.cc:374] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:45.360257 23629 raft_consensus.cc:729] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089726e841e944828f84758edd5b33ef, State: Initialized, Role: FOLLOWER
I20260806 00:47:45.360257 23630 raft_consensus.cc:729] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 897b9a79dd094ed081dd24b4ab0d0162, State: Initialized, Role: FOLLOWER
I20260806 00:47:45.360734 23629 consensus_queue.cc:260] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } }
I20260806 00:47:45.360734 23630 consensus_queue.cc:260] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } }
I20260806 00:47:45.361440 23420 heartbeater.cc:503] Master 127.29.84.190:40683 was elected leader, sending a full tablet report...
I20260806 00:47:45.361737 23232 heartbeater.cc:503] Master 127.29.84.190:40683 was elected leader, sending a full tablet report...
I20260806 00:47:45.361758 23629 ts_tablet_manager.cc:1428] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:47:45.361776 23632 raft_consensus.cc:348] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } }
I20260806 00:47:45.362001 23632 raft_consensus.cc:374] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:45.362061 23632 raft_consensus.cc:729] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bb1dfb4cde943e6ba3d4b251a3b2b8e, State: Initialized, Role: FOLLOWER
I20260806 00:47:45.362453 23632 consensus_queue.cc:260] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } }
I20260806 00:47:45.362799 23630 ts_tablet_manager.cc:1428] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 00:47:45.363399 23632 ts_tablet_manager.cc:1428] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:47:45.363457 23606 heartbeater.cc:503] Master 127.29.84.190:40683 was elected leader, sending a full tablet report...
I20260806 00:47:45.393088 23641 raft_consensus.cc:482] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:45.393285 23641 raft_consensus.cc:504] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } }
I20260806 00:47:45.394362 23641 leader_election.cc:288] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 089726e841e944828f84758edd5b33ef (127.29.84.129:35653), 2bb1dfb4cde943e6ba3d4b251a3b2b8e (127.29.84.131:39361)
I20260806 00:47:45.399278 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.394766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:49179 (local address 127.29.84.131:39361)
0806 00:47:45.394908 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.394913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.395115 (+ 202us) server_negotiation.cc:407] Connection header received
0806 00:47:45.395265 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.395268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.395306 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.395365 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:45.395702 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.396386 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.397883 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.398485 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.398603 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.398651 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.399084 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.399155 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:47:45.399278 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.394654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:45377 (local address 127.29.84.129:35653)
0806 00:47:45.394774 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.394780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.394793 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:47:45.394937 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.394941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.394996 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.395054 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:45.395458 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.396212 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.397874 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.398482 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.398602 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.398651 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.399085 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.399158 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:47:45.399365 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.394521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:35653 (local address 127.29.84.130:45377)
0806 00:47:45.394711 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.394727 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:45.394809 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:45.395130 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:45.395133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:45.395163 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:45.395310 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.395317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.396320 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.396322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.397714 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.397726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.398622 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.398626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.398896 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.398938 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 00:47:45.399031 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:47:45.399196 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":88,"spinlock_wait_cycles":9728}
I20260806 00:47:45.399418 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.394658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:39361 (local address 127.29.84.130:49179)
0806 00:47:45.395000 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.395013 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:45.395161 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:45.395461 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:45.395469 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:45.395487 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:45.395593 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.395598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.396483 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.396486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.397767 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.397779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.398622 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.398626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.398918 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.398958 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 00:47:45.399031 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:47:45.399196 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":222,"spinlock_wait_cycles":768,"thread_start_us":109,"threads_started":1}
I20260806 00:47:45.400153 23166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04181b8d57cd4fa8af511d5ace231a6e" candidate_uuid: "897b9a79dd094ed081dd24b4ab0d0162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089726e841e944828f84758edd5b33ef" is_pre_election: true
I20260806 00:47:45.400154 23543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04181b8d57cd4fa8af511d5ace231a6e" candidate_uuid: "897b9a79dd094ed081dd24b4ab0d0162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" is_pre_election: true
I20260806 00:47:45.400461 23166 raft_consensus.cc:2454] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 897b9a79dd094ed081dd24b4ab0d0162 in term 0.
I20260806 00:47:45.400482 23543 raft_consensus.cc:2454] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 897b9a79dd094ed081dd24b4ab0d0162 in term 0.
I20260806 00:47:45.400987 23269 leader_election.cc:302] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 089726e841e944828f84758edd5b33ef, 2bb1dfb4cde943e6ba3d4b251a3b2b8e, 897b9a79dd094ed081dd24b4ab0d0162; no voters:
I20260806 00:47:45.401280 23641 raft_consensus.cc:2781] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:47:45.401398 23641 raft_consensus.cc:482] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:47:45.401500 23641 raft_consensus.cc:3037] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:45.402500 23641 raft_consensus.cc:504] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } }
I20260806 00:47:45.402818 23641 leader_election.cc:288] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [CANDIDATE]: Term 1 election: Requested vote from peers 089726e841e944828f84758edd5b33ef (127.29.84.129:35653), 2bb1dfb4cde943e6ba3d4b251a3b2b8e (127.29.84.131:39361)
I20260806 00:47:45.403223 23543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04181b8d57cd4fa8af511d5ace231a6e" candidate_uuid: "897b9a79dd094ed081dd24b4ab0d0162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e"
I20260806 00:47:45.403232 23166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04181b8d57cd4fa8af511d5ace231a6e" candidate_uuid: "897b9a79dd094ed081dd24b4ab0d0162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089726e841e944828f84758edd5b33ef"
I20260806 00:47:45.403357 23166 raft_consensus.cc:3037] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:45.403357 23543 raft_consensus.cc:3037] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:45.404771 23543 raft_consensus.cc:2454] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 897b9a79dd094ed081dd24b4ab0d0162 in term 1.
I20260806 00:47:45.404771 23166 raft_consensus.cc:2454] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 897b9a79dd094ed081dd24b4ab0d0162 in term 1.
I20260806 00:47:45.405153 23269 leader_election.cc:302] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bb1dfb4cde943e6ba3d4b251a3b2b8e, 897b9a79dd094ed081dd24b4ab0d0162; no voters:
I20260806 00:47:45.405336 23641 raft_consensus.cc:2781] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:45.405557 23641 raft_consensus.cc:686] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 1 LEADER]: Becoming Leader. State: Replica: 897b9a79dd094ed081dd24b4ab0d0162, State: Running, Role: LEADER
I20260806 00:47:45.405905 23641 consensus_queue.cc:237] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } }
I20260806 00:47:45.409029 22960 catalog_manager.cc:5495] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 reported cstate change: term changed from 0 to 1, leader changed from <none> to 897b9a79dd094ed081dd24b4ab0d0162 (127.29.84.130). New cstate: current_term: 1 leader_uuid: "897b9a79dd094ed081dd24b4ab0d0162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "897b9a79dd094ed081dd24b4ab0d0162" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 45165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 } health_report { overall_health: UNKNOWN } } }
I20260806 00:47:45.432195 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.428831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.29.84.130:45165)
0806 00:47:45.428964 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.428968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.428997 (+ 29us) server_negotiation.cc:407] Connection header received
0806 00:47:45.429145 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.429147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.429194 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.429275 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:47:45.429508 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.429971 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.430877 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.431094 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.431190 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.431483 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:47:45.431828 (+ 345us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:47:45.431911 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.431973 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.432014 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:47:45.437558 23166 raft_consensus.cc:1261] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [term 1 FOLLOWER]: Refusing update from remote peer 897b9a79dd094ed081dd24b4ab0d0162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:45.437559 23543 raft_consensus.cc:1261] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [term 1 FOLLOWER]: Refusing update from remote peer 897b9a79dd094ed081dd24b4ab0d0162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:45.438261 23644 consensus_queue.cc:1035] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb1dfb4cde943e6ba3d4b251a3b2b8e" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 39361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:45.438450 23641 consensus_queue.cc:1035] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089726e841e944828f84758edd5b33ef" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 35653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:45.445632 23647 mvcc.cc:205] Tried to move back new op lower bound from 7315362879226118144 to 7315362879105032192. Current Snapshot: MvccSnapshot[applied={T|T < 7315362879226118144}]
I20260806 00:47:45.446421 23649 mvcc.cc:205] Tried to move back new op lower bound from 7315362879226118144 to 7315362879105032192. Current Snapshot: MvccSnapshot[applied={T|T < 7315362879226118144}]
I20260806 00:47:45.448261 23650 mvcc.cc:205] Tried to move back new op lower bound from 7315362879226118144 to 7315362879105032192. Current Snapshot: MvccSnapshot[applied={T|T < 7315362879226118144}]
I20260806 00:47:45.453399 22961 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:45.453534 22961 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:45.455832 22961 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdf0a3177ac843b3ba14bbe7bdb03c1b: Sending DeleteTablet for 3 replicas of tablet 04181b8d57cd4fa8af511d5ace231a6e
I20260806 00:47:45.456579 23140 tablet_service.cc:1514] Processing DeleteTablet for tablet 04181b8d57cd4fa8af511d5ace231a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:45 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:38568
I20260806 00:47:45.457041 23660 tablet_replica.cc:330] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef: stopping tablet replica
I20260806 00:47:45.457047 23326 tablet_service.cc:1514] Processing DeleteTablet for tablet 04181b8d57cd4fa8af511d5ace231a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:45 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:47324
I20260806 00:47:45.457326 23660 raft_consensus.cc:2229] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:45.457523 23661 tablet_replica.cc:330] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162: stopping tablet replica
I20260806 00:47:45.457612 23660 raft_consensus.cc:2258] T 04181b8d57cd4fa8af511d5ace231a6e P 089726e841e944828f84758edd5b33ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:45.457651 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 23042
I20260806 00:47:45.462936 23515 tablet_service.cc:1514] Processing DeleteTablet for tablet 04181b8d57cd4fa8af511d5ace231a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:45 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:52446
I20260806 00:47:45.463415 23662 tablet_replica.cc:330] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: stopping tablet replica
I20260806 00:47:45.463560 23661 raft_consensus.cc:2229] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:47:45.463663 23662 raft_consensus.cc:2229] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:45.463912 23662 raft_consensus.cc:2258] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:45.464078 23661 raft_consensus.cc:2258] T 04181b8d57cd4fa8af511d5ace231a6e P 897b9a79dd094ed081dd24b4ab0d0162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:45.465269 23662 ts_tablet_manager.cc:1905] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:45.467947 23662 ts_tablet_manager.cc:1918] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:45.468040 23662 log.cc:1196] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/04181b8d57cd4fa8af511d5ace231a6e
I20260806 00:47:45.468351 23662 ts_tablet_manager.cc:1939] T 04181b8d57cd4fa8af511d5ace231a6e P 2bb1dfb4cde943e6ba3d4b251a3b2b8e: Deleting consensus metadata
W20260806 00:47:45.468564 22932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:35653 (error 108)
I20260806 00:47:45.469350 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 23238
I20260806 00:47:45.469410 22934 catalog_manager.cc:4841] TS 2bb1dfb4cde943e6ba3d4b251a3b2b8e (127.29.84.131:39361): tablet 04181b8d57cd4fa8af511d5ace231a6e (table test-table [id=965abf157f2d4c53829585c7b80ab1bc]) successfully deleted
I20260806 00:47:45.476130 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.475794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:35653 (local address 127.0.0.1:38578)
0806 00:47:45.475973 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.476041 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:35653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260806 00:47:45.476604 22932 catalog_manager.cc:4584] TS 089726e841e944828f84758edd5b33ef (127.29.84.129:35653): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 04181b8d57cd4fa8af511d5ace231a6e: Network error: Client connection negotiation failed: client connection to 127.29.84.129:35653: connect: Connection refused (error 111)
I20260806 00:47:45.481498 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 23426
I20260806 00:47:45.481832 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.481598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:45165 (local address 127.0.0.1:47338)
0806 00:47:45.481694 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.481754 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.130:45165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 00:47:45.489184 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 22910
2026-08-06T00:47:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2406 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T00:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:45Z Disabled control of system clock
I20260806 00:47:45.522511 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43605
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:43605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:47:45.630946 23668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:45.631211 23668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:45.631246 23668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:45.635231 23668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:45.635295 23668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:45.635310 23668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:45.635326 23668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:45.635341 23668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:45.640509 23668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:43605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:45.641534 23668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:45.642890 23668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:45.650864 23678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:45.650908 23674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:45.650898 23681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:45.651122 23668 server_base.cc:1034] running on GCE node
I20260806 00:47:45.652027 23668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:45.653020 23668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:45.654208 23668 hybrid_clock.cc:648] HybridClock initialized: now 1785977265654194 us; error 25 us; skew 500 ppm
I20260806 00:47:45.655949 23668 webserver.cc:460] Webserver started at http://127.29.84.190:37167/ using document root <none> and password file <none>
I20260806 00:47:45.656522 23668 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:45.656616 23668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:45.656834 23668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:45.658751 23668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "faf18966d987f7d7d75a5891578319db"
server_key_iv: "41d7f004e7b5785d2b3ede7f18f444de"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.659344 23668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "faf18966d987f7d7d75a5891578319db"
server_key_iv: "41d7f004e7b5785d2b3ede7f18f444de"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.662768 23668 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260806 00:47:45.665057 23688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.666136 23668 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:45.666261 23668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "faf18966d987f7d7d75a5891578319db"
server_key_iv: "41d7f004e7b5785d2b3ede7f18f444de"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.666378 23668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:45.681471 23668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:45.682066 23668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:45.682250 23668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:45.689133 23668 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:43605
I20260806 00:47:45.689165 23759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:43605 every 8 connection(s)
I20260806 00:47:45.689944 23668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:45.692803 23762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:45.698009 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 23668
I20260806 00:47:45.698004 23762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c: Bootstrap starting.
I20260806 00:47:45.698159 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:45.698444 30034 external_mini_cluster.cc:1428] Setting key d0dba34cf3adddfdfd7072bb7da933f1
I20260806 00:47:45.700577 23762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:45.701289 23762 log.cc:824] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:45.703162 23762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c: No bootstrap required, opened a new log
I20260806 00:47:45.707199 23762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43605 } }
I20260806 00:47:45.707417 23762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:45.707484 23765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.699770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59784 (local address 127.29.84.190:43605)
0806 00:47:45.700360 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.700374 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.700407 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:47:45.701037 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.701054 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.701484 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.701807 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:45.702216 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.703194 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.704386 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.704817 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.704934 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.705742 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:45.705747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:45.705757 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:45.706113 (+ 356us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:45.706166 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:45.706185 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:45.706258 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.706438 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.706697 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":130,"threads_started":1}
I20260806 00:47:45.707505 23762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60e58f7a1257481ab9ff2fbc94e4ba2c, State: Initialized, Role: FOLLOWER
I20260806 00:47:45.708031 23762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43605 } }
I20260806 00:47:45.708184 23762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:45.708245 23762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:45.708343 23762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:45.709162 23762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43605 } }
I20260806 00:47:45.709546 23762 leader_election.cc:302] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60e58f7a1257481ab9ff2fbc94e4ba2c; no voters:
I20260806 00:47:45.709862 23762 leader_election.cc:288] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:47:45.709930 23768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:45.710186 23768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [term 1 LEADER]: Becoming Leader. State: Replica: 60e58f7a1257481ab9ff2fbc94e4ba2c, State: Running, Role: LEADER
I20260806 00:47:45.710458 23768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43605 } }
I20260806 00:47:45.710951 23762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:45.712096 23770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60e58f7a1257481ab9ff2fbc94e4ba2c. Latest consensus state: current_term: 1 leader_uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43605 } } }
I20260806 00:47:45.712214 23770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:45.712572 23780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:45.712536 23769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e58f7a1257481ab9ff2fbc94e4ba2c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43605 } } }
I20260806 00:47:45.715198 23769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:45.715881 23780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:45.721025 23780 catalog_manager.cc:1347] Generated new cluster ID: 69c72464ca4e4bcfafdd960c2b61a3fa
I20260806 00:47:45.721091 23780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:45.729094 23780 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:47:45.730067 23780 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:45.735940 23780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c: Generated new TSK 0
I20260806 00:47:45.736490 23780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:47:45.740324 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:43605
--builtin_ntp_servers=127.29.84.148:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:47:45.848368 23796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:45.848639 23796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:45.848697 23796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:45.852511 23796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:45.852552 23796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:45.852640 23796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:47:45.857244 23796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:43605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:45.858359 23796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:45.860050 23796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:45.869177 23805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:45.869189 23806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:45.869189 23809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:45.869419 23796 server_base.cc:1034] running on GCE node
I20260806 00:47:45.869859 23796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:45.870522 23796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:45.871701 23796 hybrid_clock.cc:648] HybridClock initialized: now 1785977265871688 us; error 27 us; skew 500 ppm
I20260806 00:47:45.873605 23796 webserver.cc:460] Webserver started at http://127.29.84.129:39023/ using document root <none> and password file <none>
I20260806 00:47:45.874150 23796 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:45.874239 23796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:45.874469 23796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:45.876423 23796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "d2e548c377a9461b8aa1543a166e0365"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "64b90c5608d138f097bb31ffc069e13c"
server_key_iv: "9e321d98be5e7041814f6bfe9b8884d8"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.876996 23796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "d2e548c377a9461b8aa1543a166e0365"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "64b90c5608d138f097bb31ffc069e13c"
server_key_iv: "9e321d98be5e7041814f6bfe9b8884d8"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.880538 23796 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:47:45.882815 23815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.884004 23796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:45.884133 23796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "d2e548c377a9461b8aa1543a166e0365"
format_stamp: "Formatted at 2026-08-06 00:47:45 on dist-test-slave-n326"
server_key: "64b90c5608d138f097bb31ffc069e13c"
server_key_iv: "9e321d98be5e7041814f6bfe9b8884d8"
server_key_version: "encryptionkey@0"
I20260806 00:47:45.884239 23796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:45.900799 23796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:45.901382 23796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:45.901587 23796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:45.902108 23796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:45.902964 23796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:45.903036 23796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.903137 23796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:45.903183 23796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:45.911954 23796 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:34745
I20260806 00:47:45.912002 23991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:34745 every 8 connection(s)
I20260806 00:47:45.912853 23796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:47:45.916965 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 23796
I20260806 00:47:45.917124 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:47:45.917464 30034 external_mini_cluster.cc:1428] Setting key 4e93267c22fb12dabd911bd5ea43cb16
I20260806 00:47:45.919446 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:43605
--builtin_ntp_servers=127.29.84.148:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:47:45.922207 23765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.914637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:54611 (local address 127.29.84.190:43605)
0806 00:47:45.914771 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:45.914775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:45.915749 (+ 974us) server_negotiation.cc:407] Connection header received
0806 00:47:45.917089 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:45.917094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:45.917156 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:45.917260 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:45.918184 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.918906 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.920134 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.920540 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.920658 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.921682 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:45.921688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:45.921693 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:45.921710 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:45.921737 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:45.921750 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:45.921830 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:45.921975 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 00:47:45.922043 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:47:45.923099 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:45.914990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:43605 (local address 127.29.84.129:54611)
0806 00:47:45.915628 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:45.915666 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:45.916856 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:45.917408 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:45.917415 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:45.917493 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:45.918019 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.918038 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.919036 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.919039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.920010 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:45.920020 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:45.920694 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:45.920697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:45.920827 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:45.921403 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:47:45.921405 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:45.921466 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:47:45.921527 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:45.921846 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:45.921853 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:45.921857 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:47:45.921953 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 00:47:45.922153 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":100,"threads_started":1}
I20260806 00:47:45.924259 23993 heartbeater.cc:347] Connected to a master server at 127.29.84.190:43605
I20260806 00:47:45.924502 23993 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:45.924989 23993 heartbeater.cc:511] Master 127.29.84.190:43605 requested a full tablet report, sending...
I20260806 00:47:45.926429 23713 ts_manager.cc:194] Registered new tserver with Master: d2e548c377a9461b8aa1543a166e0365 (127.29.84.129:34745)
I20260806 00:47:45.927649 23713 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.129:54611
W20260806 00:47:46.027931 23996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:46.028185 23996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:46.028252 23996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:46.032045 23996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:46.032104 23996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:46.032186 23996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:47:46.036536 23996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:43605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:46.037453 23996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:46.038678 23996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:46.047430 24006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:46.047407 24008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:46.047526 24005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:46.047766 23996 server_base.cc:1034] running on GCE node
I20260806 00:47:46.048136 23996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:46.048684 23996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:46.049855 23996 hybrid_clock.cc:648] HybridClock initialized: now 1785977266049841 us; error 27 us; skew 500 ppm
I20260806 00:47:46.051506 23996 webserver.cc:460] Webserver started at http://127.29.84.130:43439/ using document root <none> and password file <none>
I20260806 00:47:46.052027 23996 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:46.052111 23996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:46.052330 23996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:46.054122 23996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "3a7613ef953a49c1997ba672412933f2"
format_stamp: "Formatted at 2026-08-06 00:47:46 on dist-test-slave-n326"
server_key: "0f30465d573cab164ace399b856c9c7d"
server_key_iv: "1bb311f25d164b6ced32dbb64ecf1580"
server_key_version: "encryptionkey@0"
I20260806 00:47:46.054637 23996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "3a7613ef953a49c1997ba672412933f2"
format_stamp: "Formatted at 2026-08-06 00:47:46 on dist-test-slave-n326"
server_key: "0f30465d573cab164ace399b856c9c7d"
server_key_iv: "1bb311f25d164b6ced32dbb64ecf1580"
server_key_version: "encryptionkey@0"
I20260806 00:47:46.058049 23996 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 00:47:46.060366 24015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:46.061501 23996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:47:46.061625 23996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "3a7613ef953a49c1997ba672412933f2"
format_stamp: "Formatted at 2026-08-06 00:47:46 on dist-test-slave-n326"
server_key: "0f30465d573cab164ace399b856c9c7d"
server_key_iv: "1bb311f25d164b6ced32dbb64ecf1580"
server_key_version: "encryptionkey@0"
I20260806 00:47:46.061726 23996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:46.078397 23996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:46.078940 23996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:46.079128 23996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:46.079623 23996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:46.080463 23996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:46.080559 23996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:46.080621 23996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:46.080675 23996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:46.089592 23996 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:38477
I20260806 00:47:46.089617 24185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:38477 every 8 connection(s)
I20260806 00:47:46.090335 23996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:47:46.096879 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 23996
I20260806 00:47:46.097023 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:47:46.097340 30034 external_mini_cluster.cc:1428] Setting key 251a6c777d16813c60e413b1af46b657
I20260806 00:47:46.099005 23765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.091988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:51535 (local address 127.29.84.190:43605)
0806 00:47:46.092115 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:46.092119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:46.093029 (+ 910us) server_negotiation.cc:407] Connection header received
0806 00:47:46.094233 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:46.094235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:46.094278 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:46.094349 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:46.095153 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.095681 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.096733 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.097087 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.097204 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.098520 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:46.098523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:46.098526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:46.098538 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:46.098559 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:46.098570 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:46.098664 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:46.098812 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 00:47:46.098863 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:47:46.099525 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:43605
--builtin_ntp_servers=127.29.84.148:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:47:46.099972 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.092308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:43605 (local address 127.29.84.130:51535)
0806 00:47:46.092891 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:46.092933 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:46.094069 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:46.094491 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:46.094496 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:46.094566 (+ 70us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:46.094971 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.094983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.095787 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.095789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.096593 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.096602 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.097366 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.097371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.097582 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.098275 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:47:46.098277 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:46.098329 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:47:46.098386 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:46.098664 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:46.098673 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:46.098678 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:47:46.098798 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 00:47:46.099059 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":114,"threads_started":1}
I20260806 00:47:46.100986 24186 heartbeater.cc:347] Connected to a master server at 127.29.84.190:43605
I20260806 00:47:46.101186 24186 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:46.101670 24186 heartbeater.cc:511] Master 127.29.84.190:43605 requested a full tablet report, sending...
I20260806 00:47:46.102677 23713 ts_manager.cc:194] Registered new tserver with Master: 3a7613ef953a49c1997ba672412933f2 (127.29.84.130:38477)
I20260806 00:47:46.103487 23713 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.130:51535
W20260806 00:47:46.211593 24191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:46.211843 24191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:46.211884 24191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:46.216075 24191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:46.216122 24191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:46.216173 24191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:47:46.220611 24191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:43605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:46.221431 24191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:46.222620 24191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:46.231385 24200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:46.231410 24198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:46.231410 24196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:46.231556 24191 server_base.cc:1034] running on GCE node
I20260806 00:47:46.231984 24191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:46.232555 24191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:46.233734 24191 hybrid_clock.cc:648] HybridClock initialized: now 1785977266233716 us; error 31 us; skew 500 ppm
I20260806 00:47:46.235581 24191 webserver.cc:460] Webserver started at http://127.29.84.131:43419/ using document root <none> and password file <none>
I20260806 00:47:46.236148 24191 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:46.236238 24191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:46.236461 24191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:46.238338 24191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "c73595dcf4364a549262a83b9efc31f5"
format_stamp: "Formatted at 2026-08-06 00:47:46 on dist-test-slave-n326"
server_key: "fa17baa5cb1a92852bd9ffd99737dc79"
server_key_iv: "0a393a7ef8ad6bc8dd6919e9a11f098d"
server_key_version: "encryptionkey@0"
I20260806 00:47:46.238870 24191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "c73595dcf4364a549262a83b9efc31f5"
format_stamp: "Formatted at 2026-08-06 00:47:46 on dist-test-slave-n326"
server_key: "fa17baa5cb1a92852bd9ffd99737dc79"
server_key_iv: "0a393a7ef8ad6bc8dd6919e9a11f098d"
server_key_version: "encryptionkey@0"
I20260806 00:47:46.242316 24191 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:47:46.244735 24210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:46.245878 24191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:47:46.246006 24191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "c73595dcf4364a549262a83b9efc31f5"
format_stamp: "Formatted at 2026-08-06 00:47:46 on dist-test-slave-n326"
server_key: "fa17baa5cb1a92852bd9ffd99737dc79"
server_key_iv: "0a393a7ef8ad6bc8dd6919e9a11f098d"
server_key_version: "encryptionkey@0"
I20260806 00:47:46.246109 24191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:46.257591 24191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:46.258147 24191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:46.258318 24191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:46.258816 24191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:46.259685 24191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:46.259752 24191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:46.259816 24191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:46.259860 24191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:46.268615 24191 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:36039
I20260806 00:47:46.268653 24390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:36039 every 8 connection(s)
I20260806 00:47:46.269369 24191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:47:46.277259 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 24191
I20260806 00:47:46.277371 23765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.270941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:42461 (local address 127.29.84.190:43605)
0806 00:47:46.271121 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:46.271126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:46.272077 (+ 951us) server_negotiation.cc:407] Connection header received
0806 00:47:46.273302 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:46.273304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:46.273349 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:46.273418 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:46.274141 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.274654 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.275734 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.275944 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.276024 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.276901 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:46.276904 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:46.276905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:46.276917 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:46.276937 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:46.276945 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:46.277002 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:46.277190 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 00:47:46.277236 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:47:46.277387 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:47:46.277705 30034 external_mini_cluster.cc:1428] Setting key d03d908fe130b8af01f3d5f3bd1df653
I20260806 00:47:46.278291 24396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.271364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:43605 (local address 127.29.84.131:42461)
0806 00:47:46.271950 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:46.271987 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:46.273132 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:46.273532 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:46.273537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:46.273608 (+ 71us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:46.274028 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.274040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.274768 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.274770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.275619 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.275629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.276063 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.276066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.276193 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.276661 (+ 468us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:47:46.276662 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:46.276698 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:47:46.276772 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:46.277039 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:46.277052 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:46.277060 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:47:46.277191 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 00:47:46.277467 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":104,"threads_started":1}
I20260806 00:47:46.279337 24392 heartbeater.cc:347] Connected to a master server at 127.29.84.190:43605
I20260806 00:47:46.279551 24392 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:46.280030 24392 heartbeater.cc:511] Master 127.29.84.190:43605 requested a full tablet report, sending...
I20260806 00:47:46.280871 23713 ts_manager.cc:194] Registered new tserver with Master: c73595dcf4364a549262a83b9efc31f5 (127.29.84.131:36039)
I20260806 00:47:46.281412 23713 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.131:42461
I20260806 00:47:46.291594 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:47:46.297216 23765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.293574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59794 (local address 127.29.84.190:43605)
0806 00:47:46.293698 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:46.293704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:46.293821 (+ 117us) server_negotiation.cc:407] Connection header received
0806 00:47:46.293916 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:46.293918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:46.293953 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:46.294014 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:46.294251 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.294757 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.295842 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.296035 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.296129 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.296834 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:46.296838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:46.296840 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:46.296851 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:46.296867 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:46.296875 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:46.296931 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:46.297002 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 00:47:46.297045 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:47:46.299886 23713 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:47:46.302663 23713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:47:46.316694 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.311650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:34745 (local address 127.0.0.1:37868)
0806 00:47:46.311954 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:46.311967 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:46.312155 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:46.312699 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:46.312705 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:46.312726 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:46.312806 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.312814 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.314095 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.314098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.315450 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.315458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.316194 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.316198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.316299 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.316334 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 00:47:46.316437 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 00:47:46.316549 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":87,"threads_started":1}
I20260806 00:47:46.316702 24411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.311321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37868 (local address 127.29.84.129:34745)
0806 00:47:46.311819 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:46.311833 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:46.312081 (+ 248us) server_negotiation.cc:407] Connection header received
0806 00:47:46.312346 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:46.312352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:46.312543 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:46.312689 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:46.312922 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.313997 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.315567 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.316075 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.316194 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.316376 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:46.316468 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 00:47:46.316556 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":114,"threads_started":1}
I20260806 00:47:46.317034 24407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.310467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:38477 (local address 127.0.0.1:36808)
0806 00:47:46.311213 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:46.311260 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:46.311394 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:46.312198 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:46.312204 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:46.312227 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:46.312338 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.312344 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.313887 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.313890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.315221 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.315231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.316502 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.316505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.316650 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.316725 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 00:47:46.316801 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 00:47:46.316895 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":122,"thread_start_us":67,"threads_started":1}
I20260806 00:47:46.317333 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.310708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.29.84.130:38477)
0806 00:47:46.311104 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:46.311118 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:46.311538 (+ 420us) server_negotiation.cc:407] Connection header received
0806 00:47:46.311646 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:46.311652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:46.311859 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:46.312023 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:46.312472 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.313761 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.315341 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.316139 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.316260 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.316433 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:46.317088 (+ 655us) server_negotiation.cc:299] Negotiation successful
0806 00:47:46.317185 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":136,"threads_started":1}
I20260806 00:47:46.317376 24408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.310508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:36039 (local address 127.0.0.1:59642)
0806 00:47:46.311365 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:46.311378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:46.311489 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:46.312527 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:46.312530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:46.312548 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:46.312636 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.312644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.313829 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.313835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.315307 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.315319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.316502 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.316505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.316678 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.316726 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 00:47:46.317218 (+ 492us) client_negotiation.cc:240] Negotiation successful
0806 00:47:46.317283 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":78,"thread_start_us":100,"threads_started":1}
I20260806 00:47:46.318063 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.311334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59642 (local address 127.29.84.131:36039)
0806 00:47:46.312073 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:46.312089 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:46.312112 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:47:46.312181 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:46.312186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:46.312347 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:46.312537 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:46.312765 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.313726 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.315412 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.316146 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.316279 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.316449 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:46.317801 (+ 1352us) server_negotiation.cc:299] Negotiation successful
0806 00:47:46.317900 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":104,"threads_started":1}
I20260806 00:47:46.319721 23887 tablet_service.cc:1467] Processing CreateTablet for tablet 881bb2d8d45f487da495c5960e11fbf3 (DEFAULT_TABLE table=test-table [id=a57b09b60da8483c887c65cce2d17fa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:46.320127 24086 tablet_service.cc:1467] Processing CreateTablet for tablet 881bb2d8d45f487da495c5960e11fbf3 (DEFAULT_TABLE table=test-table [id=a57b09b60da8483c887c65cce2d17fa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:46.320873 23887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 881bb2d8d45f487da495c5960e11fbf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:46.321111 24086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 881bb2d8d45f487da495c5960e11fbf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:46.321382 24286 tablet_service.cc:1467] Processing CreateTablet for tablet 881bb2d8d45f487da495c5960e11fbf3 (DEFAULT_TABLE table=test-table [id=a57b09b60da8483c887c65cce2d17fa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:46.322255 24286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 881bb2d8d45f487da495c5960e11fbf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:46.328717 24414 tablet_bootstrap.cc:492] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: Bootstrap starting.
I20260806 00:47:46.328758 24415 tablet_bootstrap.cc:492] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365: Bootstrap starting.
I20260806 00:47:46.329159 24416 tablet_bootstrap.cc:492] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: Bootstrap starting.
I20260806 00:47:46.330593 24414 tablet_bootstrap.cc:654] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:46.330862 24415 tablet_bootstrap.cc:654] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:46.330873 24416 tablet_bootstrap.cc:654] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:46.331439 24414 log.cc:824] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:46.331719 24416 log.cc:824] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:46.331719 24415 log.cc:824] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:46.333432 24415 tablet_bootstrap.cc:492] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365: No bootstrap required, opened a new log
I20260806 00:47:46.333451 24416 tablet_bootstrap.cc:492] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: No bootstrap required, opened a new log
I20260806 00:47:46.333456 24414 tablet_bootstrap.cc:492] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: No bootstrap required, opened a new log
I20260806 00:47:46.333623 24414 ts_tablet_manager.cc:1397] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 00:47:46.333623 24416 ts_tablet_manager.cc:1397] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 00:47:46.333752 24415 ts_tablet_manager.cc:1397] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:47:46.336851 24415 raft_consensus.cc:348] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } }
I20260806 00:47:46.337090 24415 raft_consensus.cc:374] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:46.337144 24415 raft_consensus.cc:729] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e548c377a9461b8aa1543a166e0365, State: Initialized, Role: FOLLOWER
I20260806 00:47:46.337568 24415 consensus_queue.cc:260] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } }
I20260806 00:47:46.337760 24414 raft_consensus.cc:348] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } }
I20260806 00:47:46.337760 24416 raft_consensus.cc:348] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } }
I20260806 00:47:46.338006 24416 raft_consensus.cc:374] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:46.338006 24414 raft_consensus.cc:374] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:46.338068 24416 raft_consensus.cc:729] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7613ef953a49c1997ba672412933f2, State: Initialized, Role: FOLLOWER
I20260806 00:47:46.338068 24414 raft_consensus.cc:729] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73595dcf4364a549262a83b9efc31f5, State: Initialized, Role: FOLLOWER
I20260806 00:47:46.338348 23993 heartbeater.cc:503] Master 127.29.84.190:43605 was elected leader, sending a full tablet report...
I20260806 00:47:46.338500 24414 consensus_queue.cc:260] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } }
I20260806 00:47:46.338500 24416 consensus_queue.cc:260] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } }
I20260806 00:47:46.338585 24415 ts_tablet_manager.cc:1428] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:47:46.339401 24416 ts_tablet_manager.cc:1428] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260806 00:47:46.339401 24414 ts_tablet_manager.cc:1428] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:47:46.339722 24392 heartbeater.cc:503] Master 127.29.84.190:43605 was elected leader, sending a full tablet report...
I20260806 00:47:46.339761 24186 heartbeater.cc:503] Master 127.29.84.190:43605 was elected leader, sending a full tablet report...
W20260806 00:47:46.341249 24187 tablet.cc:2369] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:47:46.403987 24424 raft_consensus.cc:482] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:46.404222 24424 raft_consensus.cc:504] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } }
I20260806 00:47:46.405325 24424 leader_election.cc:288] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c73595dcf4364a549262a83b9efc31f5 (127.29.84.131:36039), d2e548c377a9461b8aa1543a166e0365 (127.29.84.129:34745)
I20260806 00:47:46.410184 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.405646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:36039 (local address 127.29.84.130:58011)
0806 00:47:46.405789 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:46.405809 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:46.405923 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:46.406319 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:46.406322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:46.406340 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:46.406495 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.406507 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.407653 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.407656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.408903 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.408914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.409784 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.409786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.409868 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.409888 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:47:46.409951 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 00:47:46.410041 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":39}
I20260806 00:47:46.410244 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.405739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:58011 (local address 127.29.84.131:36039)
0806 00:47:46.405881 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:46.405887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:46.405910 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:47:46.406067 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:46.406070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:46.406120 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:46.406187 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:46.406628 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.407547 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.409031 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.409647 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.409770 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.409823 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:46.409999 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 00:47:46.410083 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:47:46.410671 24411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.406124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:58973 (local address 127.29.84.129:34745)
0806 00:47:46.406244 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:46.406248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:46.406536 (+ 288us) server_negotiation.cc:407] Connection header received
0806 00:47:46.406712 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:46.406715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:46.406789 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:46.406879 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:46.407165 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.407868 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.409240 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.409907 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.410050 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.410094 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:46.410427 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 00:47:46.410538 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:47:46.410918 24313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "881bb2d8d45f487da495c5960e11fbf3" candidate_uuid: "3a7613ef953a49c1997ba672412933f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73595dcf4364a549262a83b9efc31f5" is_pre_election: true
I20260806 00:47:46.411034 24425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.406008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:34745 (local address 127.29.84.130:58973)
0806 00:47:46.406449 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:46.406465 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:46.406601 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:46.406897 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:46.406899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:46.406915 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:46.407015 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.407019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.407979 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.407982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.409132 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:46.409139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.410059 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.410062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:46.410154 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.410175 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:47:46.410811 (+ 636us) client_negotiation.cc:240] Negotiation successful
0806 00:47:46.410880 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":24,"thread_start_us":144,"threads_started":1}
I20260806 00:47:46.411247 24313 raft_consensus.cc:2454] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a7613ef953a49c1997ba672412933f2 in term 0.
I20260806 00:47:46.411695 23911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "881bb2d8d45f487da495c5960e11fbf3" candidate_uuid: "3a7613ef953a49c1997ba672412933f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e548c377a9461b8aa1543a166e0365" is_pre_election: true
I20260806 00:47:46.411777 24017 leader_election.cc:302] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7613ef953a49c1997ba672412933f2, c73595dcf4364a549262a83b9efc31f5; no voters:
I20260806 00:47:46.411993 23911 raft_consensus.cc:2454] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a7613ef953a49c1997ba672412933f2 in term 0.
I20260806 00:47:46.412124 24424 raft_consensus.cc:2781] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:47:46.412243 24424 raft_consensus.cc:482] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:47:46.412302 24424 raft_consensus.cc:3037] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:46.413515 24424 raft_consensus.cc:504] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } }
I20260806 00:47:46.413816 24424 leader_election.cc:288] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [CANDIDATE]: Term 1 election: Requested vote from peers c73595dcf4364a549262a83b9efc31f5 (127.29.84.131:36039), d2e548c377a9461b8aa1543a166e0365 (127.29.84.129:34745)
W20260806 00:47:46.414003 23994 tablet.cc:2369] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:47:46.414217 24313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "881bb2d8d45f487da495c5960e11fbf3" candidate_uuid: "3a7613ef953a49c1997ba672412933f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73595dcf4364a549262a83b9efc31f5"
I20260806 00:47:46.414338 24313 raft_consensus.cc:3037] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:46.415205 23911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "881bb2d8d45f487da495c5960e11fbf3" candidate_uuid: "3a7613ef953a49c1997ba672412933f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e548c377a9461b8aa1543a166e0365"
I20260806 00:47:46.415344 23911 raft_consensus.cc:3037] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:46.415660 24313 raft_consensus.cc:2454] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a7613ef953a49c1997ba672412933f2 in term 1.
I20260806 00:47:46.415969 24017 leader_election.cc:302] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7613ef953a49c1997ba672412933f2, c73595dcf4364a549262a83b9efc31f5; no voters:
I20260806 00:47:46.416185 24424 raft_consensus.cc:2781] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:46.416414 24424 raft_consensus.cc:686] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 1 LEADER]: Becoming Leader. State: Replica: 3a7613ef953a49c1997ba672412933f2, State: Running, Role: LEADER
I20260806 00:47:46.416687 24424 consensus_queue.cc:237] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } }
I20260806 00:47:46.416766 23911 raft_consensus.cc:2454] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a7613ef953a49c1997ba672412933f2 in term 1.
I20260806 00:47:46.419850 23712 catalog_manager.cc:5495] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a7613ef953a49c1997ba672412933f2 (127.29.84.130). New cstate: current_term: 1 leader_uuid: "3a7613ef953a49c1997ba672412933f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7613ef953a49c1997ba672412933f2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 38477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 } health_report { overall_health: UNKNOWN } } }
I20260806 00:47:46.437630 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.434328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.29.84.130:38477)
0806 00:47:46.434460 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:46.434464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:46.434479 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:47:46.434625 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:46.434627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:46.434675 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:46.434767 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:47:46.434952 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.435449 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.436300 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:46.436535 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:46.436624 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:46.436898 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:47:46.437262 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:47:46.437353 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:46.437410 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 00:47:46.437454 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:47:46.443468 23911 raft_consensus.cc:1261] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365 [term 1 FOLLOWER]: Refusing update from remote peer 3a7613ef953a49c1997ba672412933f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:46.443503 24313 raft_consensus.cc:1261] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [term 1 FOLLOWER]: Refusing update from remote peer 3a7613ef953a49c1997ba672412933f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:46.444136 24426 consensus_queue.cc:1035] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e548c377a9461b8aa1543a166e0365" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:46.444298 24424 consensus_queue.cc:1035] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73595dcf4364a549262a83b9efc31f5" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 36039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:46.451889 24431 mvcc.cc:205] Tried to move back new op lower bound from 7315362883343822848 to 7315362883245531136. Current Snapshot: MvccSnapshot[applied={T|T < 7315362883343822848}]
I20260806 00:47:46.453217 24430 mvcc.cc:205] Tried to move back new op lower bound from 7315362883343822848 to 7315362883245531136. Current Snapshot: MvccSnapshot[applied={T|T < 7315362883343822848}]
I20260806 00:47:46.458984 23711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:46.459179 23711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:46.461326 23711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60e58f7a1257481ab9ff2fbc94e4ba2c: Sending DeleteTablet for 3 replicas of tablet 881bb2d8d45f487da495c5960e11fbf3
I20260806 00:47:46.462080 24086 tablet_service.cc:1514] Processing DeleteTablet for tablet 881bb2d8d45f487da495c5960e11fbf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:46 UTC) from {username='slave'} at 127.0.0.1:36808
I20260806 00:47:46.462157 23887 tablet_service.cc:1514] Processing DeleteTablet for tablet 881bb2d8d45f487da495c5960e11fbf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:46 UTC) from {username='slave'} at 127.0.0.1:37868
I20260806 00:47:46.462153 24286 tablet_service.cc:1514] Processing DeleteTablet for tablet 881bb2d8d45f487da495c5960e11fbf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:46 UTC) from {username='slave'} at 127.0.0.1:59642
I20260806 00:47:46.462630 24444 tablet_replica.cc:330] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365: stopping tablet replica
I20260806 00:47:46.462854 24443 tablet_replica.cc:330] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: stopping tablet replica
I20260806 00:47:46.462922 24444 raft_consensus.cc:2229] T 881bb2d8d45f487da495c5960e11fbf3 P d2e548c377a9461b8aa1543a166e0365 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:46.463208 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 23796
I20260806 00:47:46.465760 24445 tablet_replica.cc:330] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: stopping tablet replica
I20260806 00:47:46.466049 24445 raft_consensus.cc:2229] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:46.466382 24445 raft_consensus.cc:2258] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:46.472643 24443 raft_consensus.cc:2229] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:47:46.473619 24445 ts_tablet_manager.cc:1905] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:46.475418 24443 raft_consensus.cc:2258] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:46.476652 24445 ts_tablet_manager.cc:1918] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:46.476768 24445 log.cc:1196] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/881bb2d8d45f487da495c5960e11fbf3
I20260806 00:47:46.476954 24443 ts_tablet_manager.cc:1905] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:46.477113 24445 ts_tablet_manager.cc:1939] T 881bb2d8d45f487da495c5960e11fbf3 P c73595dcf4364a549262a83b9efc31f5: Deleting consensus metadata
I20260806 00:47:46.478276 23689 catalog_manager.cc:4841] TS c73595dcf4364a549262a83b9efc31f5 (127.29.84.131:36039): tablet 881bb2d8d45f487da495c5960e11fbf3 (table test-table [id=a57b09b60da8483c887c65cce2d17fa8]) successfully deleted
I20260806 00:47:46.480562 24443 ts_tablet_manager.cc:1918] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:46.480680 24443 log.cc:1196] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/881bb2d8d45f487da495c5960e11fbf3
I20260806 00:47:46.481021 24443 ts_tablet_manager.cc:1939] T 881bb2d8d45f487da495c5960e11fbf3 P 3a7613ef953a49c1997ba672412933f2: Deleting consensus metadata
W20260806 00:47:46.481225 23689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:34745 (error 108)
I20260806 00:47:46.482107 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 23996
I20260806 00:47:46.482174 23692 catalog_manager.cc:4841] TS 3a7613ef953a49c1997ba672412933f2 (127.29.84.130:38477): tablet 881bb2d8d45f487da495c5960e11fbf3 (table test-table [id=a57b09b60da8483c887c65cce2d17fa8]) successfully deleted
I20260806 00:47:46.492591 24408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:46.481955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:34745 (local address 127.0.0.1:37884)
0806 00:47:46.482076 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:46.482146 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:34745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 00:47:46.492779 23689 catalog_manager.cc:4584] TS d2e548c377a9461b8aa1543a166e0365 (127.29.84.129:34745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 881bb2d8d45f487da495c5960e11fbf3: Network error: Client connection negotiation failed: client connection to 127.29.84.129:34745: connect: Connection refused (error 111)
I20260806 00:47:46.497727 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 24191
I20260806 00:47:46.507220 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 23668
2026-08-06T00:47:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1017 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:47:46 dist-test-slave-n326 krb5kdc[24449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:47:46 dist-test-slave-n326 krb5kdc[24449](info): set up 2 sockets
Aug 06 00:47:46 dist-test-slave-n326 krb5kdc[24449](info): commencing operation
krb5kdc: starting...
W20260806 00:47:47.700155 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:47:47 dist-test-slave-n326 krb5kdc[24449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:47:47.823944 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40533
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:33949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:40533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:47.934799 24472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:47.935060 24472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:47.935146 24472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:47.938980 24472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:47.939054 24472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:47.939110 24472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:47.939167 24472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:47.939208 24472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:47.944209 24472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:40533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:47.945140 24472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:47.946079 24472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:47:47.953869 24472 server_base.cc:1034] running on GCE node
W20260806 00:47:47.953896 24481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:47.953898 24482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:47.954018 24484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:47.954684 24472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:47.955719 24472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:47.956914 24472 hybrid_clock.cc:648] HybridClock initialized: now 1785977267956899 us; error 26 us; skew 500 ppm
Aug 06 00:47:47 dist-test-slave-n326 krb5kdc[24449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977267, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:47.962870 24472 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:47:47.963712 24472 webserver.cc:460] Webserver started at http://127.29.84.190:35577/ using document root <none> and password file <none>
I20260806 00:47:47.964253 24472 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:47.964339 24472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:47.964563 24472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:47.966418 24472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "78f450f59a404d1cbb44aee45627138d"
format_stamp: "Formatted at 2026-08-06 00:47:47 on dist-test-slave-n326"
server_key: "e3071ca40e516acda93ab5e51cff41b0"
server_key_iv: "5f345197fbb8eceffe61aa06f18179c3"
server_key_version: "encryptionkey@0"
I20260806 00:47:47.966982 24472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "78f450f59a404d1cbb44aee45627138d"
format_stamp: "Formatted at 2026-08-06 00:47:47 on dist-test-slave-n326"
server_key: "e3071ca40e516acda93ab5e51cff41b0"
server_key_iv: "5f345197fbb8eceffe61aa06f18179c3"
server_key_version: "encryptionkey@0"
I20260806 00:47:47.970289 24472 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:47:47.972477 24493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:47.973522 24472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:47:47.973654 24472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "78f450f59a404d1cbb44aee45627138d"
format_stamp: "Formatted at 2026-08-06 00:47:47 on dist-test-slave-n326"
server_key: "e3071ca40e516acda93ab5e51cff41b0"
server_key_iv: "5f345197fbb8eceffe61aa06f18179c3"
server_key_version: "encryptionkey@0"
I20260806 00:47:47.973759 24472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:47.994186 24472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:47.996766 24472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:47.997071 24472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:48.003774 24472 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:40533
I20260806 00:47:48.003808 24571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:40533 every 8 connection(s)
I20260806 00:47:48.004670 24472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:48.007491 24572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:48.009347 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 24472
I20260806 00:47:48.009510 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:48.009869 30034 external_mini_cluster.cc:1428] Setting key c92d368e247b40e783109fcf36d56b9a
I20260806 00:47:48.014005 24572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d: Bootstrap starting.
I20260806 00:47:48.016270 24572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:48.017205 24572 log.cc:824] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:47:48 dist-test-slave-n326 krb5kdc[24449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:48.019266 24572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d: No bootstrap required, opened a new log
I20260806 00:47:48.023025 24572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f450f59a404d1cbb44aee45627138d" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40533 } }
I20260806 00:47:48.023329 24572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:48.023411 24572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78f450f59a404d1cbb44aee45627138d, State: Initialized, Role: FOLLOWER
I20260806 00:47:48.023881 24572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f450f59a404d1cbb44aee45627138d" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40533 } }
I20260806 00:47:48.023972 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.011291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57248 (local address 127.29.84.190:40533)
0806 00:47:48.011746 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:48.011759 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:48.011791 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:47:48.012402 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:48.012417 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:48.012738 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:48.013074 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:48.013931 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.014966 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.015854 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.016297 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.016492 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.019314 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:48.019319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:48.019330 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:48.019373 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:48.021175 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.021645 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.021651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.021655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.021757 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.022049 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.022052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.022054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.022401 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:48.022619 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:48.022893 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 00:47:48.023204 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":141,"threads_started":1}
I20260806 00:47:48.024044 24572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:48.024119 24572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:48.024221 24572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:48.025275 24572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f450f59a404d1cbb44aee45627138d" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40533 } }
I20260806 00:47:48.025599 24572 leader_election.cc:302] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78f450f59a404d1cbb44aee45627138d; no voters:
I20260806 00:47:48.025887 24572 leader_election.cc:288] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:47:48.025990 24579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:48.026187 24579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [term 1 LEADER]: Becoming Leader. State: Replica: 78f450f59a404d1cbb44aee45627138d, State: Running, Role: LEADER
I20260806 00:47:48.026481 24579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f450f59a404d1cbb44aee45627138d" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40533 } }
I20260806 00:47:48.027149 24572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:48.027817 24582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78f450f59a404d1cbb44aee45627138d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f450f59a404d1cbb44aee45627138d" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40533 } } }
I20260806 00:47:48.027932 24582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:48.028324 24588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:48.028290 24583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78f450f59a404d1cbb44aee45627138d. Latest consensus state: current_term: 1 leader_uuid: "78f450f59a404d1cbb44aee45627138d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f450f59a404d1cbb44aee45627138d" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40533 } } }
I20260806 00:47:48.028429 24583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:48.031401 24588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:48.036628 24588 catalog_manager.cc:1347] Generated new cluster ID: 4c1dde17e57b489e9fcfb362fbeb1db6
I20260806 00:47:48.036700 24588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:48.058825 24588 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:47:48.059725 24588 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:48.075114 24588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d: Generated new TSK 0
I20260806 00:47:48.075754 24588 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:47:48.106103 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40533
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:33949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:48.218609 24616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:48.218825 24616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:48.218864 24616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:48.222749 24616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:48.222800 24616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:48.222852 24616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:47:48.228025 24616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:48.228914 24616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:48.229866 24616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:48.238833 24625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:48.238792 24623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:48.238869 24616 server_base.cc:1034] running on GCE node
W20260806 00:47:48.238790 24622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:48.239395 24616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:48.239948 24616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:48.241099 24616 hybrid_clock.cc:648] HybridClock initialized: now 1785977268241086 us; error 26 us; skew 500 ppm
Aug 06 00:47:48 dist-test-slave-n326 krb5kdc[24449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977268, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:48.243716 24616 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:47:48.244429 24616 webserver.cc:460] Webserver started at http://127.29.84.129:42055/ using document root <none> and password file <none>
I20260806 00:47:48.244915 24616 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:48.244971 24616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:48.245119 24616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:48.246944 24616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444"
format_stamp: "Formatted at 2026-08-06 00:47:48 on dist-test-slave-n326"
server_key: "804c22cc33e24271922319e78de892ec"
server_key_iv: "92c3df1b7b325ed9f15bc8c6efeb3801"
server_key_version: "encryptionkey@0"
I20260806 00:47:48.247488 24616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444"
format_stamp: "Formatted at 2026-08-06 00:47:48 on dist-test-slave-n326"
server_key: "804c22cc33e24271922319e78de892ec"
server_key_iv: "92c3df1b7b325ed9f15bc8c6efeb3801"
server_key_version: "encryptionkey@0"
I20260806 00:47:48.250865 24616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:47:48.253079 24639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:48.254209 24616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:48.254323 24616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444"
format_stamp: "Formatted at 2026-08-06 00:47:48 on dist-test-slave-n326"
server_key: "804c22cc33e24271922319e78de892ec"
server_key_iv: "92c3df1b7b325ed9f15bc8c6efeb3801"
server_key_version: "encryptionkey@0"
I20260806 00:47:48.254416 24616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:48.269076 24616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:48.271548 24616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:48.271718 24616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:48.272220 24616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:48.272936 24616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:48.272981 24616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:48.273047 24616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:48.273078 24616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:48.281690 24616 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:40161
I20260806 00:47:48.281723 24811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:40161 every 8 connection(s)
I20260806 00:47:48.282454 24616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
Aug 06 00:47:48 dist-test-slave-n326 krb5kdc[24449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977268, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:48.291739 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 24616
I20260806 00:47:48.291879 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:47:48.292227 30034 external_mini_cluster.cc:1428] Setting key aa6608e619c8685bb80933cda7c2b8c6
I20260806 00:47:48.295408 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.284075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:51187 (local address 127.29.84.190:40533)
0806 00:47:48.284195 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:48.284199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:48.284931 (+ 732us) server_negotiation.cc:407] Connection header received
0806 00:47:48.286209 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:48.286213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:48.286271 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:48.286367 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:48.287475 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.288005 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.289091 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.289317 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.289422 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.291664 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:48.291668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:48.291671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:48.291703 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:48.293193 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.293796 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.293800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.293802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.293894 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.294214 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.294220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.294225 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.294468 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:48.294583 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:48.294993 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 00:47:48.295167 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:47:48.295928 24817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.284392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40533 (local address 127.29.84.129:51187)
0806 00:47:48.284804 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:48.284841 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:48.286016 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:48.286533 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:48.286540 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:48.286846 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:48.287338 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.287351 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.288142 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.288146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.288962 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.288971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.289457 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.289462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.289613 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.290133 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:48.290148 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:48.291510 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:48.293302 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:48.293308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:48.293320 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:48.293695 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:48.293994 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:48.293996 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:48.293997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:48.294120 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:48.294580 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:48.294584 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:48.294778 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 00:47:48.294920 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 00:47:48.295195 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":121,"threads_started":1}
I20260806 00:47:48.297032 24812 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40533
I20260806 00:47:48.297251 24812 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:48.297675 24812 heartbeater.cc:511] Master 127.29.84.190:40533 requested a full tablet report, sending...
I20260806 00:47:48.298990 24518 ts_manager.cc:194] Registered new tserver with Master: 1937c70b3e8b4ed9a6c4ab428c6b8444 (127.29.84.129:40161)
I20260806 00:47:48.300071 24518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:51187
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:47:48.319977 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40533
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:33949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:48.430830 24820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:48.431118 24820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:48.431183 24820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:48.435524 24820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:48.435571 24820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:48.435626 24820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:47:48.440482 24820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:48.441298 24820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:48.442175 24820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:48.450907 24832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:48.450907 24830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:48.450887 24829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:48.451130 24820 server_base.cc:1034] running on GCE node
I20260806 00:47:48.451684 24820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:48.452194 24820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:48.453346 24820 hybrid_clock.cc:648] HybridClock initialized: now 1785977268453322 us; error 35 us; skew 500 ppm
Aug 06 00:47:48 dist-test-slave-n326 krb5kdc[24449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977268, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:48.455996 24820 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:47:48.456815 24820 webserver.cc:460] Webserver started at http://127.29.84.130:44423/ using document root <none> and password file <none>
I20260806 00:47:48.457401 24820 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:48.457484 24820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:48.457695 24820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:48.459663 24820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "b7b38bbe62834901a190fc587dc52bb2"
format_stamp: "Formatted at 2026-08-06 00:47:48 on dist-test-slave-n326"
server_key: "4ecc8eb248b3ca50f670b1d51212db43"
server_key_iv: "7774254671255bea22533f8188e3c7ab"
server_key_version: "encryptionkey@0"
I20260806 00:47:48.460211 24820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "b7b38bbe62834901a190fc587dc52bb2"
format_stamp: "Formatted at 2026-08-06 00:47:48 on dist-test-slave-n326"
server_key: "4ecc8eb248b3ca50f670b1d51212db43"
server_key_iv: "7774254671255bea22533f8188e3c7ab"
server_key_version: "encryptionkey@0"
I20260806 00:47:48.463639 24820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:47:48.465855 24841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:48.466899 24820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:48.467047 24820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "b7b38bbe62834901a190fc587dc52bb2"
format_stamp: "Formatted at 2026-08-06 00:47:48 on dist-test-slave-n326"
server_key: "4ecc8eb248b3ca50f670b1d51212db43"
server_key_iv: "7774254671255bea22533f8188e3c7ab"
server_key_version: "encryptionkey@0"
I20260806 00:47:48.467188 24820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:48.480557 24820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:48.483042 24820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:48.483266 24820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:48.483767 24820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:48.484488 24820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:48.484561 24820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:48.484642 24820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:48.484686 24820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:48.493119 24820 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:43319
I20260806 00:47:48.493144 25022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:43319 every 8 connection(s)
I20260806 00:47:48.493847 24820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:47:48.495469 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 24820
I20260806 00:47:48.495617 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:47:48.495947 30034 external_mini_cluster.cc:1428] Setting key 64e6a4986299e07adc5a9bff3838f169
Aug 06 00:47:48 dist-test-slave-n326 krb5kdc[24449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977268, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:47:48.507588 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.495378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:60531 (local address 127.29.84.190:40533)
0806 00:47:48.495498 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:48.495502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:48.496502 (+ 1000us) server_negotiation.cc:407] Connection header received
0806 00:47:48.497812 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:48.497818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:48.497889 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:48.497983 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:48.498986 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.499688 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.501060 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.501291 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.501419 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.504142 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:48.504145 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:48.504147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:48.504170 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:48.505503 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.506091 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.506094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.506095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.506164 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.506540 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.506542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.506543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.506695 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:48.506766 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:48.507379 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 00:47:48.507476 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:47:48.508397 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.495864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40533 (local address 127.29.84.130:60531)
0806 00:47:48.496337 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:48.496390 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:48.497618 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:48.498140 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:48.498146 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:48.498431 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:48.498825 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.498837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.499806 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.499811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.500942 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.500958 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.501470 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.501474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.501633 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.502148 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:48.502163 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:48.503978 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:48.505617 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:48.505622 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:48.505636 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:48.505977 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:48.506253 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:48.506257 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:48.506260 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:48.506437 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:48.506801 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:48.506807 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:48.507117 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 00:47:48.507341 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 00:47:48.507652 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":142,"threads_started":1}
I20260806 00:47:48.509400 25024 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40533
I20260806 00:47:48.509629 25024 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
I20260806 00:47:48.510216 25024 heartbeater.cc:511] Master 127.29.84.190:40533 requested a full tablet report, sending...
I20260806 00:47:48.511282 24518 ts_manager.cc:194] Registered new tserver with Master: b7b38bbe62834901a190fc587dc52bb2 (127.29.84.130:43319)
I20260806 00:47:48.512004 24518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:60531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:47:48.526058 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40533
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:33949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:48.632441 25030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:48.632665 25030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:48.632704 25030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:48.636384 25030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:48.636433 25030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:48.636485 25030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:47:48.641356 25030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:40533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:48.642215 25030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:48.643149 25030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:48.652180 25041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:48.652169 25042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:48.652182 25046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:48.652354 25030 server_base.cc:1034] running on GCE node
I20260806 00:47:48.652805 25030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:48.653506 25030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:48.654692 25030 hybrid_clock.cc:648] HybridClock initialized: now 1785977268654676 us; error 32 us; skew 500 ppm
Aug 06 00:47:48 dist-test-slave-n326 krb5kdc[24449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977268, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:48.657431 25030 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:47:48.658247 25030 webserver.cc:460] Webserver started at http://127.29.84.131:42183/ using document root <none> and password file <none>
I20260806 00:47:48.658799 25030 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:48.658886 25030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:48.659176 25030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:48.661103 25030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "2172c207c02d4566891012ad458a8432"
format_stamp: "Formatted at 2026-08-06 00:47:48 on dist-test-slave-n326"
server_key: "7ed39eb81d8b7f010aeabd0cdeaca467"
server_key_iv: "4f8ba669faab3ae88b2ad54df1d8e99d"
server_key_version: "encryptionkey@0"
I20260806 00:47:48.661677 25030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "2172c207c02d4566891012ad458a8432"
format_stamp: "Formatted at 2026-08-06 00:47:48 on dist-test-slave-n326"
server_key: "7ed39eb81d8b7f010aeabd0cdeaca467"
server_key_iv: "4f8ba669faab3ae88b2ad54df1d8e99d"
server_key_version: "encryptionkey@0"
I20260806 00:47:48.665166 25030 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:47:48.667321 25058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:48.668457 25030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:47:48.668596 25030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "2172c207c02d4566891012ad458a8432"
format_stamp: "Formatted at 2026-08-06 00:47:48 on dist-test-slave-n326"
server_key: "7ed39eb81d8b7f010aeabd0cdeaca467"
server_key_iv: "4f8ba669faab3ae88b2ad54df1d8e99d"
server_key_version: "encryptionkey@0"
I20260806 00:47:48.668694 25030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:48.681912 25030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:48.684722 25030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:48.684916 25030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:48.685416 25030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:48.686173 25030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:48.686240 25030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:48.686311 25030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:48.686352 25030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:48.694933 25030 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:44483
I20260806 00:47:48.694989 25237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:44483 every 8 connection(s)
I20260806 00:47:48.695734 25030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:47:48.701784 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 25030
I20260806 00:47:48.701936 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:47:48.702227 30034 external_mini_cluster.cc:1428] Setting key 54f9b49237a1552b20c09726f4868e4d
Aug 06 00:47:48 dist-test-slave-n326 krb5kdc[24449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977268, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:48.708338 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.697324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:46909 (local address 127.29.84.190:40533)
0806 00:47:48.697465 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:48.697469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:48.698224 (+ 755us) server_negotiation.cc:407] Connection header received
0806 00:47:48.699452 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:48.699456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:48.699515 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:48.699591 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:48.700594 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.701115 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.702320 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.702605 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.702717 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.705300 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:48.705302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:48.705304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:48.705328 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:48.706652 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.707143 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.707145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.707146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.707221 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.707506 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.707509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.707512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.707669 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:48.707739 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:48.708135 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 00:47:48.708232 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:47:48.709013 25244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.697621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:40533 (local address 127.29.84.131:46909)
0806 00:47:48.698055 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:48.698095 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:48.699250 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:48.699738 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:48.699744 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:48.700041 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:48.700446 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.700459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.701241 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.701244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.702195 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.702205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.702888 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.702892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.703104 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.703691 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:48.703710 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:48.705143 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:48.706745 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:48.706748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:48.706757 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:48.707018 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:48.707295 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:48.707297 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:48.707298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:48.707421 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:48.707749 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:48.707753 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:48.707946 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 00:47:48.708079 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 00:47:48.708292 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":103,"threads_started":1}
I20260806 00:47:48.709894 25238 heartbeater.cc:347] Connected to a master server at 127.29.84.190:40533
I20260806 00:47:48.710089 25238 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:48.710506 25238 heartbeater.cc:511] Master 127.29.84.190:40533 requested a full tablet report, sending...
I20260806 00:47:48.711536 24518 ts_manager.cc:194] Registered new tserver with Master: 2172c207c02d4566891012ad458a8432 (127.29.84.131:44483)
I20260806 00:47:48.712127 24518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:46909
I20260806 00:47:48.715847 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:47:48.724483 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.717729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.29.84.190:40533)
0806 00:47:48.717860 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:48.717865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:48.718059 (+ 194us) server_negotiation.cc:407] Connection header received
0806 00:47:48.718200 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:48.718202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:48.718233 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:48.718290 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:48.718935 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.719435 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.720202 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.720389 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.720457 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.721723 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:48.721726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:48.721727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:48.721748 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:48.723102 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.723536 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.723538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.723539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.723601 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:48.723836 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:48.723838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:48.723839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:48.723986 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:48.724096 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:48.724294 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 00:47:48.724357 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:47:48.726994 24518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:47:48.730124 24518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:47:48.743798 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.737678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:40161 (local address 127.0.0.1:42332)
0806 00:47:48.738282 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:48.738326 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:48.738981 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:48.739696 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:48.739699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:48.739719 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:48.739808 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.739814 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.741061 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.741066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.742141 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.742150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.743269 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.743273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.743381 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.743420 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 00:47:48.743557 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 00:47:48.743664 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":4,"thread_start_us":76,"threads_started":1}
I20260806 00:47:48.743846 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.737804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42332 (local address 127.29.84.129:40161)
0806 00:47:48.738740 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:48.738752 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:48.739100 (+ 348us) server_negotiation.cc:407] Connection header received
0806 00:47:48.739186 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:48.739191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:48.739371 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:48.739526 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:48.739936 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.740937 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.742260 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.743143 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.743271 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.743414 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:48.743593 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 00:47:48.743699 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":121,"threads_started":1}
I20260806 00:47:48.744077 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.738393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.29.84.131:44483)
0806 00:47:48.738750 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:48.738762 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:48.738784 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:47:48.738852 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:48.738859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:48.739050 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:48.739222 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:48.739609 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.740676 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.742418 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.743226 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.743312 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.743413 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:48.743869 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 00:47:48.743961 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":97,"threads_started":1}
I20260806 00:47:48.744328 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.737777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:44483 (local address 127.0.0.1:48188)
0806 00:47:48.738299 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:48.738326 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:48.738439 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:48.739344 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:48.739349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:48.739366 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:48.739479 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.739486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.740810 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.740813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.742288 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.742300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.743488 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.743490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.743622 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.743645 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:47:48.743715 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 00:47:48.743780 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":161,"thread_start_us":104,"threads_started":1}
I20260806 00:47:48.744939 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.738107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:43319 (local address 127.0.0.1:60960)
0806 00:47:48.738899 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:48.738916 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:48.738985 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:48.739507 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:48.739512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:48.739533 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:48.739627 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.739632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.740625 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.740631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.742288 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:48.742300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.744417 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.744420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:48.744509 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.744538 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 00:47:48.744756 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 00:47:48.744825 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":165,"thread_start_us":94,"threads_started":1}
I20260806 00:47:48.745030 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:48.738485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.29.84.130:43319)
0806 00:47:48.738836 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:48.738846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:48.739106 (+ 260us) server_negotiation.cc:407] Connection header received
0806 00:47:48.739189 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:48.739192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:48.739371 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:48.739517 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:48.739727 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.740492 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.743176 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:48.744090 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:48.744201 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:48.744349 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:48.744801 (+ 452us) server_negotiation.cc:299] Negotiation successful
0806 00:47:48.744898 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":86,"threads_started":1}
I20260806 00:47:48.746783 24711 tablet_service.cc:1467] Processing CreateTablet for tablet c0c9131835d44830ad3efaf5e9c9c5fc (DEFAULT_TABLE table=test-table [id=c9e03617ce464b3d835a5b0fdbdeb245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:48.746969 25137 tablet_service.cc:1467] Processing CreateTablet for tablet c0c9131835d44830ad3efaf5e9c9c5fc (DEFAULT_TABLE table=test-table [id=c9e03617ce464b3d835a5b0fdbdeb245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:48.747361 24911 tablet_service.cc:1467] Processing CreateTablet for tablet c0c9131835d44830ad3efaf5e9c9c5fc (DEFAULT_TABLE table=test-table [id=c9e03617ce464b3d835a5b0fdbdeb245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:48.747867 24711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0c9131835d44830ad3efaf5e9c9c5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:48.747959 25137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0c9131835d44830ad3efaf5e9c9c5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:48.748252 24911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0c9131835d44830ad3efaf5e9c9c5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:48.753978 25266 tablet_bootstrap.cc:492] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444: Bootstrap starting.
I20260806 00:47:48.755255 25269 tablet_bootstrap.cc:492] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: Bootstrap starting.
I20260806 00:47:48.755569 25268 tablet_bootstrap.cc:492] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: Bootstrap starting.
I20260806 00:47:48.756314 25266 tablet_bootstrap.cc:654] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:48.757460 25268 tablet_bootstrap.cc:654] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:48.757450 25269 tablet_bootstrap.cc:654] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:48.757710 25266 log.cc:824] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:48.758648 25269 log.cc:824] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:48.758741 25268 log.cc:824] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:48.759811 25266 tablet_bootstrap.cc:492] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444: No bootstrap required, opened a new log
I20260806 00:47:48.759999 25266 ts_tablet_manager.cc:1397] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:47:48.760481 25269 tablet_bootstrap.cc:492] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: No bootstrap required, opened a new log
I20260806 00:47:48.760638 25269 ts_tablet_manager.cc:1397] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:47:48.763809 25266 raft_consensus.cc:348] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } }
I20260806 00:47:48.764060 25266 raft_consensus.cc:374] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:48.764114 25266 raft_consensus.cc:729] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1937c70b3e8b4ed9a6c4ab428c6b8444, State: Initialized, Role: FOLLOWER
I20260806 00:47:48.764223 25268 tablet_bootstrap.cc:492] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: No bootstrap required, opened a new log
I20260806 00:47:48.764230 25269 raft_consensus.cc:348] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } }
I20260806 00:47:48.764408 25268 ts_tablet_manager.cc:1397] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260806 00:47:48.764458 25269 raft_consensus.cc:374] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:48.764520 25269 raft_consensus.cc:729] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2172c207c02d4566891012ad458a8432, State: Initialized, Role: FOLLOWER
I20260806 00:47:48.764575 25266 consensus_queue.cc:260] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } }
I20260806 00:47:48.764909 25269 consensus_queue.cc:260] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } }
I20260806 00:47:48.765264 24812 heartbeater.cc:503] Master 127.29.84.190:40533 was elected leader, sending a full tablet report...
I20260806 00:47:48.765558 25266 ts_tablet_manager.cc:1428] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260806 00:47:48.765774 25269 ts_tablet_manager.cc:1428] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:47:48.766070 25238 heartbeater.cc:503] Master 127.29.84.190:40533 was elected leader, sending a full tablet report...
I20260806 00:47:48.768361 25268 raft_consensus.cc:348] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } }
I20260806 00:47:48.768642 25268 raft_consensus.cc:374] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:48.768712 25268 raft_consensus.cc:729] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b38bbe62834901a190fc587dc52bb2, State: Initialized, Role: FOLLOWER
I20260806 00:47:48.769161 25268 consensus_queue.cc:260] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } }
I20260806 00:47:48.769825 25024 heartbeater.cc:503] Master 127.29.84.190:40533 was elected leader, sending a full tablet report...
I20260806 00:47:48.770102 25268 ts_tablet_manager.cc:1428] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 00:47:48.783496 24813 tablet.cc:2369] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:47:48.946588 25242 tablet.cc:2369] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:47:48.994879 25025 tablet.cc:2369] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:47:49.046233 25275 raft_consensus.cc:482] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:49.046439 25275 raft_consensus.cc:504] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } }
I20260806 00:47:49.047570 25275 leader_election.cc:288] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2172c207c02d4566891012ad458a8432 (127.29.84.131:44483), b7b38bbe62834901a190fc587dc52bb2 (127.29.84.130:43319)
I20260806 00:47:49.052923 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.047882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:56677 (local address 127.29.84.131:44483)
0806 00:47:49.048014 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.048020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.048248 (+ 228us) server_negotiation.cc:407] Connection header received
0806 00:47:49.048389 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.048392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.048464 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.048568 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:49.049016 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.049797 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.051699 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.052294 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.052409 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.052457 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:49.052711 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 00:47:49.052796 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:47:49.052923 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.047896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:42589 (local address 127.29.84.130:43319)
0806 00:47:49.048025 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.048029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.048250 (+ 221us) server_negotiation.cc:407] Connection header received
0806 00:47:49.048451 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.048455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.048512 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.048589 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:49.049002 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.049797 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.051699 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.052294 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.052409 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.052457 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:49.052710 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 00:47:49.052796 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:47:49.052983 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.047797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:43319 (local address 127.29.84.129:42589)
0806 00:47:49.048172 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:49.048188 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:49.048307 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:49.048602 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:49.048605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:49.048648 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:49.048883 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.048893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.049915 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.049919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.051585 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.051594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.052409 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.052412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.052544 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.052614 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 00:47:49.052687 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:47:49.052798 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":346,"spinlock_wait_cycles":4608,"thread_start_us":66,"threads_started":1}
I20260806 00:47:49.052986 25280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.047782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:44483 (local address 127.29.84.129:56677)
0806 00:47:49.048138 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:49.048153 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:49.048277 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:49.048602 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:49.048605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:49.048647 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:49.048893 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.048899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.049915 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.049919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.051583 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.051594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.052409 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.052412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.052554 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.052617 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 00:47:49.052687 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 00:47:49.052798 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":250,"spinlock_wait_cycles":37376,"thread_start_us":162,"threads_started":1}
I20260806 00:47:49.053768 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c9131835d44830ad3efaf5e9c9c5fc" candidate_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b38bbe62834901a190fc587dc52bb2" is_pre_election: true
I20260806 00:47:49.053771 25170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c9131835d44830ad3efaf5e9c9c5fc" candidate_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2172c207c02d4566891012ad458a8432" is_pre_election: true
I20260806 00:47:49.054018 25170 raft_consensus.cc:2454] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1937c70b3e8b4ed9a6c4ab428c6b8444 in term 0.
I20260806 00:47:49.054033 24941 raft_consensus.cc:2454] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1937c70b3e8b4ed9a6c4ab428c6b8444 in term 0.
I20260806 00:47:49.054484 24647 leader_election.cc:302] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1937c70b3e8b4ed9a6c4ab428c6b8444, b7b38bbe62834901a190fc587dc52bb2; no voters:
I20260806 00:47:49.054741 25275 raft_consensus.cc:2781] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:47:49.054826 25275 raft_consensus.cc:482] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:47:49.054888 25275 raft_consensus.cc:3037] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:49.055938 25275 raft_consensus.cc:504] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } }
I20260806 00:47:49.056314 25275 leader_election.cc:288] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [CANDIDATE]: Term 1 election: Requested vote from peers 2172c207c02d4566891012ad458a8432 (127.29.84.131:44483), b7b38bbe62834901a190fc587dc52bb2 (127.29.84.130:43319)
I20260806 00:47:49.056604 25170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c9131835d44830ad3efaf5e9c9c5fc" candidate_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2172c207c02d4566891012ad458a8432"
I20260806 00:47:49.056689 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c9131835d44830ad3efaf5e9c9c5fc" candidate_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b38bbe62834901a190fc587dc52bb2"
I20260806 00:47:49.056710 25170 raft_consensus.cc:3037] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:49.056825 24941 raft_consensus.cc:3037] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:49.057893 25170 raft_consensus.cc:2454] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1937c70b3e8b4ed9a6c4ab428c6b8444 in term 1.
I20260806 00:47:49.057971 24941 raft_consensus.cc:2454] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1937c70b3e8b4ed9a6c4ab428c6b8444 in term 1.
I20260806 00:47:49.058207 24644 leader_election.cc:302] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1937c70b3e8b4ed9a6c4ab428c6b8444, 2172c207c02d4566891012ad458a8432; no voters:
I20260806 00:47:49.058373 25275 raft_consensus.cc:2781] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:49.058605 25275 raft_consensus.cc:686] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [term 1 LEADER]: Becoming Leader. State: Replica: 1937c70b3e8b4ed9a6c4ab428c6b8444, State: Running, Role: LEADER
I20260806 00:47:49.058905 25275 consensus_queue.cc:237] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } }
I20260806 00:47:49.061692 24518 catalog_manager.cc:5495] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1937c70b3e8b4ed9a6c4ab428c6b8444 (127.29.84.129). New cstate: current_term: 1 leader_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1937c70b3e8b4ed9a6c4ab428c6b8444" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 40161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260806 00:47:49.104856 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.101363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.29.84.129:40161)
0806 00:47:49.101495 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.101504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.101532 (+ 28us) server_negotiation.cc:407] Connection header received
0806 00:47:49.101719 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.101722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.101782 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.101900 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:47:49.102202 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.102765 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.103584 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.103787 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.103860 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.104144 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:47:49.104478 (+ 334us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:47:49.104584 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:49.104668 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 00:47:49.104722 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:47:49.110052 24941 raft_consensus.cc:1261] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [term 1 FOLLOWER]: Refusing update from remote peer 1937c70b3e8b4ed9a6c4ab428c6b8444: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:49.110052 25170 raft_consensus.cc:1261] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [term 1 FOLLOWER]: Refusing update from remote peer 1937c70b3e8b4ed9a6c4ab428c6b8444: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:49.110675 25275 consensus_queue.cc:1035] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b38bbe62834901a190fc587dc52bb2" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:49.110878 25282 consensus_queue.cc:1035] T c0c9131835d44830ad3efaf5e9c9c5fc P 1937c70b3e8b4ed9a6c4ab428c6b8444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2172c207c02d4566891012ad458a8432" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 44483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:49.118793 25289 mvcc.cc:205] Tried to move back new op lower bound from 7315362894268739584 to 7315362894067417088. Current Snapshot: MvccSnapshot[applied={T|T < 7315362894268739584}]
I20260806 00:47:49.126106 24516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:49.126259 24516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:49.128597 24516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78f450f59a404d1cbb44aee45627138d: Sending DeleteTablet for 3 replicas of tablet c0c9131835d44830ad3efaf5e9c9c5fc
I20260806 00:47:49.129277 25137 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c9131835d44830ad3efaf5e9c9c5fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:49 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:48188
I20260806 00:47:49.129377 24911 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c9131835d44830ad3efaf5e9c9c5fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:49 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:60960
I20260806 00:47:49.129706 25300 tablet_replica.cc:330] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: stopping tablet replica
I20260806 00:47:49.129715 24711 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c9131835d44830ad3efaf5e9c9c5fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:49 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:42332
I20260806 00:47:49.130017 25300 raft_consensus.cc:2229] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:49.130298 25300 raft_consensus.cc:2258] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:49.130304 25301 tablet_replica.cc:330] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: stopping tablet replica
I20260806 00:47:49.130570 25301 raft_consensus.cc:2229] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:49.130826 25301 raft_consensus.cc:2258] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:49.130892 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 24616
I20260806 00:47:49.134788 25300 ts_tablet_manager.cc:1905] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:49.138525 25300 ts_tablet_manager.cc:1918] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:49.138641 25300 log.cc:1196] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/c0c9131835d44830ad3efaf5e9c9c5fc
I20260806 00:47:49.139091 25300 ts_tablet_manager.cc:1939] T c0c9131835d44830ad3efaf5e9c9c5fc P 2172c207c02d4566891012ad458a8432: Deleting consensus metadata
I20260806 00:47:49.141467 24499 catalog_manager.cc:4841] TS 2172c207c02d4566891012ad458a8432 (127.29.84.131:44483): tablet c0c9131835d44830ad3efaf5e9c9c5fc (table test-table [id=c9e03617ce464b3d835a5b0fdbdeb245]) successfully deleted
I20260806 00:47:49.142876 25301 ts_tablet_manager.cc:1905] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:49.145480 25301 ts_tablet_manager.cc:1918] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:49.145582 25301 log.cc:1196] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/c0c9131835d44830ad3efaf5e9c9c5fc
I20260806 00:47:49.145884 25301 ts_tablet_manager.cc:1939] T c0c9131835d44830ad3efaf5e9c9c5fc P b7b38bbe62834901a190fc587dc52bb2: Deleting consensus metadata
I20260806 00:47:49.146898 24500 catalog_manager.cc:4841] TS b7b38bbe62834901a190fc587dc52bb2 (127.29.84.130:43319): tablet c0c9131835d44830ad3efaf5e9c9c5fc (table test-table [id=c9e03617ce464b3d835a5b0fdbdeb245]) successfully deleted
I20260806 00:47:49.149361 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 24820
W20260806 00:47:49.149363 24494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:40161 (error 108)
I20260806 00:47:49.151801 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.151382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:40161 (local address 127.0.0.1:42360)
0806 00:47:49.151604 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:49.151695 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:40161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260806 00:47:49.152040 24494 catalog_manager.cc:4584] TS 1937c70b3e8b4ed9a6c4ab428c6b8444 (127.29.84.129:40161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c0c9131835d44830ad3efaf5e9c9c5fc: Network error: Client connection negotiation failed: client connection to 127.29.84.129:40161: connect: Connection refused (error 111)
I20260806 00:47:49.159795 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 25030
I20260806 00:47:49.173974 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 24472
2026-08-06T00:47:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2667 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T00:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:49Z Disabled control of system clock
I20260806 00:47:49.208355 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:33053
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:33053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:47:49.315025 25311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:49.315268 25311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:49.315302 25311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:49.319427 25311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:49.319481 25311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:49.319496 25311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:49.319514 25311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:49.319548 25311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:49.324270 25311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:33053
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:33053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:49.325232 25311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:49.326498 25311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:49.334685 25327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:49.334681 25323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:49.334858 25322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:49.334964 25311 server_base.cc:1034] running on GCE node
I20260806 00:47:49.335656 25311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:49.336639 25311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:49.337806 25311 hybrid_clock.cc:648] HybridClock initialized: now 1785977269337785 us; error 32 us; skew 500 ppm
I20260806 00:47:49.339629 25311 webserver.cc:460] Webserver started at http://127.29.84.190:39499/ using document root <none> and password file <none>
I20260806 00:47:49.340188 25311 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:49.340250 25311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:49.340497 25311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:49.342463 25311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "eff9401d85fe432ca2d5725d33102e5c"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "3f149a2a91b0fde0ad1fd145cdb28a13"
server_key_iv: "148614eb978cd9212951cdae7c3decdb"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.343004 25311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "eff9401d85fe432ca2d5725d33102e5c"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "3f149a2a91b0fde0ad1fd145cdb28a13"
server_key_iv: "148614eb978cd9212951cdae7c3decdb"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.346479 25311 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:47:49.348866 25335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.350054 25311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:49.350286 25311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "eff9401d85fe432ca2d5725d33102e5c"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "3f149a2a91b0fde0ad1fd145cdb28a13"
server_key_iv: "148614eb978cd9212951cdae7c3decdb"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.350392 25311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:49.366770 25311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:49.367492 25311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:49.367682 25311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:49.374305 25311 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:33053
I20260806 00:47:49.374352 25405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:33053 every 8 connection(s)
I20260806 00:47:49.375239 25311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:49.378201 25406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:49.383445 25406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c: Bootstrap starting.
I20260806 00:47:49.383754 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 25311
I20260806 00:47:49.383900 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:49.384174 30034 external_mini_cluster.cc:1428] Setting key 153eb000bb9ad7ca8735fb6fe798a039
I20260806 00:47:49.386011 25406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:49.386721 25406 log.cc:824] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:49.388568 25406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c: No bootstrap required, opened a new log
I20260806 00:47:49.391902 25406 raft_consensus.cc:348] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff9401d85fe432ca2d5725d33102e5c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33053 } }
I20260806 00:47:49.392129 25406 raft_consensus.cc:374] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:49.392197 25406 raft_consensus.cc:729] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff9401d85fe432ca2d5725d33102e5c, State: Initialized, Role: FOLLOWER
I20260806 00:47:49.392656 25406 consensus_queue.cc:260] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff9401d85fe432ca2d5725d33102e5c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33053 } }
I20260806 00:47:49.392809 25406 raft_consensus.cc:388] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:49.392894 25406 raft_consensus.cc:482] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:49.393004 25406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:49.393567 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.385480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36928 (local address 127.29.84.190:33053)
0806 00:47:49.386115 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.386131 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.386166 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:47:49.386807 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.386829 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.387336 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.387710 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:49.388118 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.389124 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.390039 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.390523 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.390688 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.391546 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:49.391556 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:49.391567 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:49.391926 (+ 359us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:49.391980 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:49.392000 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:49.392101 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:49.392290 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 00:47:49.392545 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":130,"threads_started":1}
I20260806 00:47:49.394064 25406 raft_consensus.cc:504] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff9401d85fe432ca2d5725d33102e5c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33053 } }
I20260806 00:47:49.394395 25406 leader_election.cc:302] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eff9401d85fe432ca2d5725d33102e5c; no voters:
I20260806 00:47:49.394711 25406 leader_election.cc:288] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:47:49.394825 25414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:49.394996 25414 raft_consensus.cc:686] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [term 1 LEADER]: Becoming Leader. State: Replica: eff9401d85fe432ca2d5725d33102e5c, State: Running, Role: LEADER
I20260806 00:47:49.395325 25414 consensus_queue.cc:237] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff9401d85fe432ca2d5725d33102e5c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33053 } }
I20260806 00:47:49.395932 25406 sys_catalog.cc:564] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:49.396479 25419 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader eff9401d85fe432ca2d5725d33102e5c. Latest consensus state: current_term: 1 leader_uuid: "eff9401d85fe432ca2d5725d33102e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff9401d85fe432ca2d5725d33102e5c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33053 } } }
I20260806 00:47:49.396615 25419 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:49.396951 25417 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eff9401d85fe432ca2d5725d33102e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff9401d85fe432ca2d5725d33102e5c" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 33053 } } }
I20260806 00:47:49.397078 25417 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:49.397481 25422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:49.400557 25422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:49.405411 25422 catalog_manager.cc:1347] Generated new cluster ID: 4ffd1fbcd0e448008aa919a307b74f90
I20260806 00:47:49.405473 25422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:49.415865 25422 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:47:49.416774 25422 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:49.421624 25422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c: Generated new TSK 0
I20260806 00:47:49.422297 25422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:47:49.424971 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33053
--builtin_ntp_servers=127.29.84.148:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:47:49.537621 25446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:49.537864 25446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:49.537938 25446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:49.541923 25446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:49.541992 25446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:49.542088 25446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:47:49.546542 25446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:49.547531 25446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:49.548733 25446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:49.557423 25463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:49.557464 25446 server_base.cc:1034] running on GCE node
W20260806 00:47:49.557526 25466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:49.557431 25460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:49.557978 25446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:49.558564 25446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:49.559715 25446 hybrid_clock.cc:648] HybridClock initialized: now 1785977269559709 us; error 31 us; skew 500 ppm
I20260806 00:47:49.561480 25446 webserver.cc:460] Webserver started at http://127.29.84.129:44319/ using document root <none> and password file <none>
I20260806 00:47:49.562002 25446 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:49.562065 25446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:49.562294 25446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:49.564239 25446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "65a84f954a3d4d6cb049252f79b4b8de"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "dd2247347442ed6fb85789ccbb59b65a"
server_key_iv: "dc01ff95811f52b9f33b84e64184c14e"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.564752 25446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "65a84f954a3d4d6cb049252f79b4b8de"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "dd2247347442ed6fb85789ccbb59b65a"
server_key_iv: "dc01ff95811f52b9f33b84e64184c14e"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.568107 25446 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:47:49.570222 25472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.571347 25446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:49.571501 25446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "65a84f954a3d4d6cb049252f79b4b8de"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "dd2247347442ed6fb85789ccbb59b65a"
server_key_iv: "dc01ff95811f52b9f33b84e64184c14e"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.571604 25446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:49.587133 25446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:49.587682 25446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:49.587867 25446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:49.588400 25446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:49.589222 25446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:49.589300 25446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.589377 25446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:49.589423 25446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.598274 25446 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:33219
I20260806 00:47:49.598345 25638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:33219 every 8 connection(s)
I20260806 00:47:49.599120 25446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:47:49.604866 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 25446
I20260806 00:47:49.605011 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:47:49.605300 30034 external_mini_cluster.cc:1428] Setting key f7086d1e5e68c745927da3e691739c70
I20260806 00:47:49.607347 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33053
--builtin_ntp_servers=127.29.84.148:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:47:49.608446 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.600791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:40237 (local address 127.29.84.190:33053)
0806 00:47:49.600924 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.600928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.601874 (+ 946us) server_negotiation.cc:407] Connection header received
0806 00:47:49.603278 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.603281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.603340 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.603465 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:49.604274 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.604985 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.606302 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.606572 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.606716 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.607975 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:49.607979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:49.607982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:49.607999 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:49.608017 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:49.608024 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:49.608099 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:49.608253 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 00:47:49.608310 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:47:49.609205 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.601140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33053 (local address 127.29.84.129:40237)
0806 00:47:49.601743 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:49.601780 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:49.603022 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:49.603640 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:49.603646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:49.603719 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:49.604142 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.604154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.605152 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.605155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.606196 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.606206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.606753 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.606757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.606911 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.607736 (+ 825us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:47:49.607738 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:49.607774 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:47:49.607830 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:49.608102 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:49.608114 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:49.608121 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 00:47:49.608253 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 00:47:49.608510 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":110,"threads_started":1}
I20260806 00:47:49.610419 25639 heartbeater.cc:347] Connected to a master server at 127.29.84.190:33053
I20260806 00:47:49.610630 25639 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:49.611130 25639 heartbeater.cc:511] Master 127.29.84.190:33053 requested a full tablet report, sending...
I20260806 00:47:49.612512 25358 ts_manager.cc:194] Registered new tserver with Master: 65a84f954a3d4d6cb049252f79b4b8de (127.29.84.129:33219)
I20260806 00:47:49.613588 25358 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.129:40237
W20260806 00:47:49.713514 25646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:49.713757 25646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:49.713825 25646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:49.717787 25646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:49.717872 25646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:49.717967 25646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:47:49.722374 25646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:49.723299 25646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:49.724525 25646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:49.733698 25656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:49.733707 25654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:49.733712 25653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:49.733777 25646 server_base.cc:1034] running on GCE node
I20260806 00:47:49.734371 25646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:49.734905 25646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:49.736056 25646 hybrid_clock.cc:648] HybridClock initialized: now 1785977269736047 us; error 63 us; skew 500 ppm
I20260806 00:47:49.737730 25646 webserver.cc:460] Webserver started at http://127.29.84.130:40833/ using document root <none> and password file <none>
I20260806 00:47:49.738260 25646 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:49.738319 25646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:49.738556 25646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:49.740471 25646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "ec8dc3979db441f59911f4c8c8cdb356"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "e925611f5144e5015e11ab97d654f63f"
server_key_iv: "c53dcceec6ca0a6e0bb68fda1f1a5c1d"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.741014 25646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "ec8dc3979db441f59911f4c8c8cdb356"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "e925611f5144e5015e11ab97d654f63f"
server_key_iv: "c53dcceec6ca0a6e0bb68fda1f1a5c1d"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.744491 25646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:47:49.746712 25665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.747870 25646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:49.747994 25646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "ec8dc3979db441f59911f4c8c8cdb356"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "e925611f5144e5015e11ab97d654f63f"
server_key_iv: "c53dcceec6ca0a6e0bb68fda1f1a5c1d"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.748100 25646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:49.770699 25646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:49.771327 25646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:49.771488 25646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:49.771976 25646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:49.772771 25646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:49.772833 25646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.772892 25646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:49.772928 25646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.781713 25646 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:34541
I20260806 00:47:49.781749 25827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:34541 every 8 connection(s)
I20260806 00:47:49.782536 25646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:47:49.784159 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 25646
I20260806 00:47:49.784296 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:47:49.784605 30034 external_mini_cluster.cc:1428] Setting key c30f4b357b6ecf2b743b81bdfc7edc15
I20260806 00:47:49.786533 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33053
--builtin_ntp_servers=127.29.84.148:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:47:49.792251 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.784304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:32921 (local address 127.29.84.190:33053)
0806 00:47:49.784495 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.784501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.785476 (+ 975us) server_negotiation.cc:407] Connection header received
0806 00:47:49.786915 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.786921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.786982 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.787110 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:49.787937 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.788675 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.790101 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.790475 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.790594 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.791661 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:49.791667 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:49.791672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:49.791690 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:49.791719 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:49.791732 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:49.791819 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:49.791949 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 00:47:49.792026 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:47:49.792927 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.784615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33053 (local address 127.29.84.130:32921)
0806 00:47:49.785311 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:49.785376 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:49.786703 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:49.787262 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:49.787269 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:49.787348 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:49.787795 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.787806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.788803 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.788808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.789976 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.789986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.790622 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.790624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.790748 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.791410 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:47:49.791411 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:49.791447 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:47:49.791504 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:49.791818 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:49.791825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:49.791829 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:47:49.791917 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 00:47:49.792127 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":127,"threads_started":1}
I20260806 00:47:49.794001 25828 heartbeater.cc:347] Connected to a master server at 127.29.84.190:33053
I20260806 00:47:49.794267 25828 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:49.795043 25828 heartbeater.cc:511] Master 127.29.84.190:33053 requested a full tablet report, sending...
I20260806 00:47:49.796370 25358 ts_manager.cc:194] Registered new tserver with Master: ec8dc3979db441f59911f4c8c8cdb356 (127.29.84.130:34541)
I20260806 00:47:49.797062 25358 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.130:32921
W20260806 00:47:49.896353 25834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:49.896597 25834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:49.896652 25834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:49.900692 25834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:49.900738 25834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:49.900827 25834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:47:49.905380 25834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:33053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:49.906256 25834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:49.907536 25834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:49.916122 25841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:49.916265 25846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:49.916122 25840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:49.916396 25834 server_base.cc:1034] running on GCE node
I20260806 00:47:49.916868 25834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:49.917533 25834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:49.918746 25834 hybrid_clock.cc:648] HybridClock initialized: now 1785977269918733 us; error 23 us; skew 500 ppm
I20260806 00:47:49.920665 25834 webserver.cc:460] Webserver started at http://127.29.84.131:40677/ using document root <none> and password file <none>
I20260806 00:47:49.921279 25834 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:49.921414 25834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:49.921682 25834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:49.923861 25834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "778761ae4ef54c24a9fbde0577b60505"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "f60ec1fb80d503e33388f3ba7eb2b71f"
server_key_iv: "e8e00aa958424d2d479e459c67844872"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.924433 25834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "778761ae4ef54c24a9fbde0577b60505"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "f60ec1fb80d503e33388f3ba7eb2b71f"
server_key_iv: "e8e00aa958424d2d479e459c67844872"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.928215 25834 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:47:49.930722 25855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.931879 25834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:49.932024 25834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "778761ae4ef54c24a9fbde0577b60505"
format_stamp: "Formatted at 2026-08-06 00:47:49 on dist-test-slave-n326"
server_key: "f60ec1fb80d503e33388f3ba7eb2b71f"
server_key_iv: "e8e00aa958424d2d479e459c67844872"
server_key_version: "encryptionkey@0"
I20260806 00:47:49.932127 25834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:49.946902 25834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:49.947722 25834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:49.947901 25834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:49.948587 25834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:49.949750 25834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:49.949808 25834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.949867 25834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:49.949900 25834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:49.960453 25834 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:34527
I20260806 00:47:49.960541 26035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:34527 every 8 connection(s)
I20260806 00:47:49.961300 25834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:47:49.963532 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 25834
I20260806 00:47:49.963646 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:47:49.963938 30034 external_mini_cluster.cc:1428] Setting key dc24ebd1aaff29c919a2d99054989d35
I20260806 00:47:49.970096 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.963104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:35389 (local address 127.29.84.190:33053)
0806 00:47:49.963242 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.963246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.964308 (+ 1062us) server_negotiation.cc:407] Connection header received
0806 00:47:49.965753 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.965756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.965813 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.965909 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:49.966706 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.967368 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.968462 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.968686 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.968771 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.969689 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:49.969692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:49.969694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:49.969705 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:49.969724 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:49.969732 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:49.969794 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:49.969932 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 00:47:49.969984 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:47:49.970820 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.963421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:33053 (local address 127.29.84.131:35389)
0806 00:47:49.964163 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:49.964212 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:49.965572 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:49.966080 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:49.966086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:49.966160 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:49.966584 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.966595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.967481 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.967483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.968343 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.968351 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.968809 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.968812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:49.968943 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.969453 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:47:49.969455 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:49.969491 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:47:49.969554 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:49.969807 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:49.969814 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:49.969818 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:47:49.969912 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 00:47:49.970102 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":157,"threads_started":1}
I20260806 00:47:49.971895 26036 heartbeater.cc:347] Connected to a master server at 127.29.84.190:33053
I20260806 00:47:49.972093 26036 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:49.972541 26036 heartbeater.cc:511] Master 127.29.84.190:33053 requested a full tablet report, sending...
I20260806 00:47:49.973465 25358 ts_manager.cc:194] Registered new tserver with Master: 778761ae4ef54c24a9fbde0577b60505 (127.29.84.131:34527)
I20260806 00:47:49.973989 25358 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.131:35389
I20260806 00:47:49.977747 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:47:49.982986 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.979694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36938 (local address 127.29.84.190:33053)
0806 00:47:49.979802 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.979806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.979934 (+ 128us) server_negotiation.cc:407] Connection header received
0806 00:47:49.980019 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.980021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.980057 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.980123 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:49.980345 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.980851 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.981639 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.981838 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.981918 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:49.982623 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:49.982626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:49.982628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:49.982638 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:49.982653 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:49.982660 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:49.982747 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:49.982870 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 00:47:49.982899 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:47:49.985512 25358 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:47:49.988626 25358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:47:50.002699 26058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.996346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:33219 (local address 127.0.0.1:52968)
0806 00:47:49.996954 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:49.996966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:49.997062 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:49.998910 (+ 1848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:49.998913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:49.998934 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:49.999022 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.999028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.000125 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.000131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.001115 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:50.001124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.002215 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.002219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.002307 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.002329 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:47:50.002384 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 00:47:50.002504 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":21,"thread_start_us":126,"threads_started":1}
I20260806 00:47:50.002725 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.996341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.29.84.131:34527)
0806 00:47:49.997313 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.997326 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.997350 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:47:49.997420 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.997425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.997638 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.998139 (+ 501us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:49.998278 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.999398 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.001426 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.001983 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.002113 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.002307 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:50.002460 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 00:47:50.002562 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":135,"threads_started":1}
I20260806 00:47:50.002828 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.996964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:34541 (local address 127.0.0.1:48192)
0806 00:47:49.997456 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:49.997468 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:49.997545 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:49.998235 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:49.998238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:49.998291 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:49.998400 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.998406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.999712 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.999715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.001246 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:50.001259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.002426 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.002429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.002503 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.002523 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:47:50.002602 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 00:47:50.002660 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":86,"thread_start_us":96,"threads_started":1}
I20260806 00:47:50.003046 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.997435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.29.84.130:34541)
0806 00:47:49.997746 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.997760 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.997786 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:47:49.997867 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.997886 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.998084 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.998259 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:49.998503 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.999576 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.001364 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.002090 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.002190 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.002364 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:50.002767 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 00:47:50.002886 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":110,"threads_started":1}
I20260806 00:47:50.002720 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.996232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:34527 (local address 127.0.0.1:36600)
0806 00:47:49.996800 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:49.996845 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:49.996966 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:49.997810 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:49.997818 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:49.997847 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:49.997986 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:49.997997 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:49.999523 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:49.999526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.001246 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:50.001259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.002116 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.002120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.002244 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.002284 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 00:47:50.002431 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 00:47:50.002516 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":90,"thread_start_us":71,"threads_started":1}
I20260806 00:47:50.003791 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:49.996934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.29.84.129:33219)
0806 00:47:49.997896 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:49.997909 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:49.997931 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:47:49.998534 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:49.998541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:49.998718 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:49.998907 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:49.999180 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.000017 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.001272 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.002068 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.003433 (+ 1365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.003578 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:50.003616 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 00:47:50.003676 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":857,"thread_start_us":201,"threads_started":1}
I20260806 00:47:50.006093 25541 tablet_service.cc:1467] Processing CreateTablet for tablet 818946d643194a768856f85968edf854 (DEFAULT_TABLE table=test-table [id=71c7746f6c82418a9d575dd1df15a53e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:50.006136 25734 tablet_service.cc:1467] Processing CreateTablet for tablet 818946d643194a768856f85968edf854 (DEFAULT_TABLE table=test-table [id=71c7746f6c82418a9d575dd1df15a53e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:50.007098 25541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 818946d643194a768856f85968edf854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:50.007295 25734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 818946d643194a768856f85968edf854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:50.007139 25919 tablet_service.cc:1467] Processing CreateTablet for tablet 818946d643194a768856f85968edf854 (DEFAULT_TABLE table=test-table [id=71c7746f6c82418a9d575dd1df15a53e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:50.008121 25919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 818946d643194a768856f85968edf854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:50.012769 26064 tablet_bootstrap.cc:492] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de: Bootstrap starting.
I20260806 00:47:50.014919 26069 tablet_bootstrap.cc:492] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: Bootstrap starting.
I20260806 00:47:50.015236 26064 tablet_bootstrap.cc:654] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:50.015522 26067 tablet_bootstrap.cc:492] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: Bootstrap starting.
I20260806 00:47:50.016165 26064 log.cc:824] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:50.016987 26069 tablet_bootstrap.cc:654] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:50.017597 26067 tablet_bootstrap.cc:654] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:50.017794 26069 log.cc:824] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:50.018497 26067 log.cc:824] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:50.018805 26064 tablet_bootstrap.cc:492] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de: No bootstrap required, opened a new log
I20260806 00:47:50.019004 26064 ts_tablet_manager.cc:1397] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:47:50.019841 26069 tablet_bootstrap.cc:492] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: No bootstrap required, opened a new log
I20260806 00:47:50.020022 26069 ts_tablet_manager.cc:1397] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:47:50.020874 26067 tablet_bootstrap.cc:492] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: No bootstrap required, opened a new log
I20260806 00:47:50.021072 26067 ts_tablet_manager.cc:1397] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:47:50.023030 26064 raft_consensus.cc:348] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } }
I20260806 00:47:50.023293 26064 raft_consensus.cc:374] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:50.023356 26064 raft_consensus.cc:729] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a84f954a3d4d6cb049252f79b4b8de, State: Initialized, Role: FOLLOWER
I20260806 00:47:50.023784 26064 consensus_queue.cc:260] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } }
I20260806 00:47:50.024042 26069 raft_consensus.cc:348] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } }
I20260806 00:47:50.024287 26069 raft_consensus.cc:374] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:50.024351 26069 raft_consensus.cc:729] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 778761ae4ef54c24a9fbde0577b60505, State: Initialized, Role: FOLLOWER
I20260806 00:47:50.024603 25639 heartbeater.cc:503] Master 127.29.84.190:33053 was elected leader, sending a full tablet report...
I20260806 00:47:50.024762 26064 ts_tablet_manager.cc:1428] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:47:50.024727 26069 consensus_queue.cc:260] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } }
I20260806 00:47:50.025331 26036 heartbeater.cc:503] Master 127.29.84.190:33053 was elected leader, sending a full tablet report...
I20260806 00:47:50.025426 26067 raft_consensus.cc:348] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } }
I20260806 00:47:50.025663 26069 ts_tablet_manager.cc:1428] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:47:50.025662 26067 raft_consensus.cc:374] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:50.025712 26067 raft_consensus.cc:729] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8dc3979db441f59911f4c8c8cdb356, State: Initialized, Role: FOLLOWER
I20260806 00:47:50.026080 26067 consensus_queue.cc:260] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } }
I20260806 00:47:50.026921 26067 ts_tablet_manager.cc:1428] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:47:50.026973 25828 heartbeater.cc:503] Master 127.29.84.190:33053 was elected leader, sending a full tablet report...
W20260806 00:47:50.033401 25830 tablet.cc:2369] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:47:50.093572 26076 raft_consensus.cc:482] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:50.093762 26076 raft_consensus.cc:504] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } }
I20260806 00:47:50.094748 26076 leader_election.cc:288] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 778761ae4ef54c24a9fbde0577b60505 (127.29.84.131:34527), 65a84f954a3d4d6cb049252f79b4b8de (127.29.84.129:33219)
I20260806 00:47:50.098501 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:50.095022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:51817 (local address 127.29.84.131:34527)
0806 00:47:50.095168 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:50.095174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:50.095282 (+ 108us) server_negotiation.cc:407] Connection header received
0806 00:47:50.095385 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:50.095390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:50.095440 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:50.095532 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:50.095802 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.096348 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.097547 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.097967 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.098057 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.098128 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:50.098339 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 00:47:50.098403 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:47:50.098502 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:50.094944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:34527 (local address 127.29.84.130:51817)
0806 00:47:50.095192 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:50.095209 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:50.095302 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:50.095544 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:50.095546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:50.095563 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:50.095682 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:50.095689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.096454 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.096458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.097421 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:50.097431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.098094 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.098097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.098199 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.098234 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 00:47:50.098319 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 00:47:50.098386 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260806 00:47:50.098582 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:50.095043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:38501 (local address 127.29.84.129:33219)
0806 00:47:50.095176 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:50.095181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:50.095440 (+ 259us) server_negotiation.cc:407] Connection header received
0806 00:47:50.095558 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:50.095561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:50.095607 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:50.095716 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:50.095980 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.096457 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.097639 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.097999 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.098130 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.098179 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:50.098400 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 00:47:50.098470 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:47:50.098601 26080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:50.094986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:33219 (local address 127.29.84.130:38501)
0806 00:47:50.095346 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:50.095361 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:50.095473 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:50.095728 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:50.095733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:50.095756 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:50.095876 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:50.095882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.096580 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.096583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.097469 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:50.097494 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.098140 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.098147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:50.098286 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.098314 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:47:50.098382 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 00:47:50.098441 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":118,"threads_started":1}
I20260806 00:47:50.099148 25566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "818946d643194a768856f85968edf854" candidate_uuid: "ec8dc3979db441f59911f4c8c8cdb356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a84f954a3d4d6cb049252f79b4b8de" is_pre_election: true
I20260806 00:47:50.099155 25957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "818946d643194a768856f85968edf854" candidate_uuid: "ec8dc3979db441f59911f4c8c8cdb356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "778761ae4ef54c24a9fbde0577b60505" is_pre_election: true
I20260806 00:47:50.099431 25957 raft_consensus.cc:2454] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec8dc3979db441f59911f4c8c8cdb356 in term 0.
I20260806 00:47:50.099442 25566 raft_consensus.cc:2454] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec8dc3979db441f59911f4c8c8cdb356 in term 0.
I20260806 00:47:50.099862 25666 leader_election.cc:302] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65a84f954a3d4d6cb049252f79b4b8de, ec8dc3979db441f59911f4c8c8cdb356; no voters:
I20260806 00:47:50.100088 26076 raft_consensus.cc:2781] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:47:50.100167 26076 raft_consensus.cc:482] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:47:50.100207 26076 raft_consensus.cc:3037] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 0 FOLLOWER]: Advancing to term 1
W20260806 00:47:50.100229 25641 tablet.cc:2369] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:47:50.101122 26076 raft_consensus.cc:504] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } }
I20260806 00:47:50.101471 26076 leader_election.cc:288] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [CANDIDATE]: Term 1 election: Requested vote from peers 778761ae4ef54c24a9fbde0577b60505 (127.29.84.131:34527), 65a84f954a3d4d6cb049252f79b4b8de (127.29.84.129:33219)
I20260806 00:47:50.101745 25957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "818946d643194a768856f85968edf854" candidate_uuid: "ec8dc3979db441f59911f4c8c8cdb356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "778761ae4ef54c24a9fbde0577b60505"
I20260806 00:47:50.101784 25566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "818946d643194a768856f85968edf854" candidate_uuid: "ec8dc3979db441f59911f4c8c8cdb356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a84f954a3d4d6cb049252f79b4b8de"
I20260806 00:47:50.101889 25566 raft_consensus.cc:3037] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:50.101892 25957 raft_consensus.cc:3037] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:50.102826 25957 raft_consensus.cc:2454] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8dc3979db441f59911f4c8c8cdb356 in term 1.
I20260806 00:47:50.102862 25566 raft_consensus.cc:2454] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8dc3979db441f59911f4c8c8cdb356 in term 1.
I20260806 00:47:50.103145 25669 leader_election.cc:302] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 778761ae4ef54c24a9fbde0577b60505, ec8dc3979db441f59911f4c8c8cdb356; no voters:
I20260806 00:47:50.103319 26076 raft_consensus.cc:2781] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:50.103526 26076 raft_consensus.cc:686] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 1 LEADER]: Becoming Leader. State: Replica: ec8dc3979db441f59911f4c8c8cdb356, State: Running, Role: LEADER
I20260806 00:47:50.103792 26076 consensus_queue.cc:237] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } }
I20260806 00:47:50.106421 25356 catalog_manager.cc:5495] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec8dc3979db441f59911f4c8c8cdb356 (127.29.84.130). New cstate: current_term: 1 leader_uuid: "ec8dc3979db441f59911f4c8c8cdb356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8dc3979db441f59911f4c8c8cdb356" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34541 } health_report { overall_health: HEALTHY } } }
I20260806 00:47:50.124827 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:50.121455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.29.84.130:34541)
0806 00:47:50.121588 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:50.121592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:50.121605 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:47:50.121752 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:50.121754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:50.121799 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:50.121900 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:47:50.122144 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.122597 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.123471 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:50.123673 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:50.123770 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:50.124104 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:47:50.124420 (+ 316us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:47:50.124564 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:50.124666 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 00:47:50.124696 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:47:50.129897 25957 raft_consensus.cc:1261] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [term 1 FOLLOWER]: Refusing update from remote peer ec8dc3979db441f59911f4c8c8cdb356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:50.129896 25566 raft_consensus.cc:1261] T 818946d643194a768856f85968edf854 P 65a84f954a3d4d6cb049252f79b4b8de [term 1 FOLLOWER]: Refusing update from remote peer ec8dc3979db441f59911f4c8c8cdb356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:50.130548 26081 consensus_queue.cc:1035] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65a84f954a3d4d6cb049252f79b4b8de" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 33219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:50.130704 26076 consensus_queue.cc:1035] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "778761ae4ef54c24a9fbde0577b60505" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:50.137188 26084 mvcc.cc:205] Tried to move back new op lower bound from 7315362898446041088 to 7315362898347380736. Current Snapshot: MvccSnapshot[applied={T|T < 7315362898446041088}]
I20260806 00:47:50.138769 26085 mvcc.cc:205] Tried to move back new op lower bound from 7315362898446041088 to 7315362898347380736. Current Snapshot: MvccSnapshot[applied={T|T < 7315362898446041088}]
I20260806 00:47:50.145248 25358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:50.145387 25358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:50.147679 25358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eff9401d85fe432ca2d5725d33102e5c: Sending DeleteTablet for 3 replicas of tablet 818946d643194a768856f85968edf854
I20260806 00:47:50.148425 25541 tablet_service.cc:1514] Processing DeleteTablet for tablet 818946d643194a768856f85968edf854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:50 UTC) from {username='slave'} at 127.0.0.1:52968
I20260806 00:47:50.148584 25919 tablet_service.cc:1514] Processing DeleteTablet for tablet 818946d643194a768856f85968edf854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:50 UTC) from {username='slave'} at 127.0.0.1:36600
I20260806 00:47:50.149024 26105 tablet_replica.cc:330] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: stopping tablet replica
I20260806 00:47:50.149024 25734 tablet_service.cc:1514] Processing DeleteTablet for tablet 818946d643194a768856f85968edf854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:50 UTC) from {username='slave'} at 127.0.0.1:48192
I20260806 00:47:50.149119 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 25446
I20260806 00:47:50.149288 26105 raft_consensus.cc:2229] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:50.149549 26105 raft_consensus.cc:2258] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:50.153424 26106 tablet_replica.cc:330] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: stopping tablet replica
I20260806 00:47:50.153771 26106 raft_consensus.cc:2229] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:47:50.154143 26106 raft_consensus.cc:2258] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:50.157177 26106 ts_tablet_manager.cc:1905] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:50.160089 26106 ts_tablet_manager.cc:1918] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:50.160195 26106 log.cc:1196] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/818946d643194a768856f85968edf854
I20260806 00:47:50.160326 26105 ts_tablet_manager.cc:1905] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:50.160575 26106 ts_tablet_manager.cc:1939] T 818946d643194a768856f85968edf854 P ec8dc3979db441f59911f4c8c8cdb356: Deleting consensus metadata
I20260806 00:47:50.161587 25342 catalog_manager.cc:4841] TS ec8dc3979db441f59911f4c8c8cdb356 (127.29.84.130:34541): tablet 818946d643194a768856f85968edf854 (table test-table [id=71c7746f6c82418a9d575dd1df15a53e]) successfully deleted
I20260806 00:47:50.162935 26105 ts_tablet_manager.cc:1918] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:50.163029 26105 log.cc:1196] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/818946d643194a768856f85968edf854
I20260806 00:47:50.163347 26105 ts_tablet_manager.cc:1939] T 818946d643194a768856f85968edf854 P 778761ae4ef54c24a9fbde0577b60505: Deleting consensus metadata
I20260806 00:47:50.164471 25342 catalog_manager.cc:4841] TS 778761ae4ef54c24a9fbde0577b60505 (127.29.84.131:34527): tablet 818946d643194a768856f85968edf854 (table test-table [id=71c7746f6c82418a9d575dd1df15a53e]) successfully deleted
W20260806 00:47:50.168313 25339 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:33219 (error 108)
I20260806 00:47:50.168502 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 25646
I20260806 00:47:50.175480 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:50.175222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:33219 (local address 127.0.0.1:52982)
0806 00:47:50.175347 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:50.175409 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:33219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 00:47:50.175740 25339 catalog_manager.cc:4584] TS 65a84f954a3d4d6cb049252f79b4b8de (127.29.84.129:33219): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 818946d643194a768856f85968edf854: Network error: Client connection negotiation failed: client connection to 127.29.84.129:33219: connect: Connection refused (error 111)
I20260806 00:47:50.179948 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 25834
I20260806 00:47:50.194177 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 25311
2026-08-06T00:47:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1019 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:47:50 dist-test-slave-n326 krb5kdc[26112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:47:50 dist-test-slave-n326 krb5kdc[26112](info): set up 2 sockets
Aug 06 00:47:50 dist-test-slave-n326 krb5kdc[26112](info): commencing operation
krb5kdc: starting...
W20260806 00:47:51.402529 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:47:51 dist-test-slave-n326 krb5kdc[26112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:47:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:47:51.522101 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38893
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:38893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:51.628278 26126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:51.628522 26126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:51.628559 26126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:51.632342 26126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:51.632390 26126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:51.632408 26126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:51.632463 26126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:51.632483 26126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:51.637405 26126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:38893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:38893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:51.638310 26126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:51.639285 26126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:51.647224 26135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:51.647229 26132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:51.647241 26133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:51.647444 26126 server_base.cc:1034] running on GCE node
I20260806 00:47:51.648108 26126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:51.649102 26126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:51.650302 26126 hybrid_clock.cc:648] HybridClock initialized: now 1785977271650282 us; error 42 us; skew 500 ppm
Aug 06 00:47:51 dist-test-slave-n326 krb5kdc[26112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977271, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:51.652920 26126 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:47:51.653748 26126 webserver.cc:460] Webserver started at http://127.29.84.190:36737/ using document root <none> and password file <none>
I20260806 00:47:51.654260 26126 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:51.654350 26126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:51.654567 26126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:51.656445 26126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "3b50a399fed04839b8e99f4cbe9ab1eb"
format_stamp: "Formatted at 2026-08-06 00:47:51 on dist-test-slave-n326"
server_key: "38503d19b626252223c458e37b76dd24"
server_key_iv: "cc02be0b13bddb8d6267a2e9f32f12e7"
server_key_version: "encryptionkey@0"
I20260806 00:47:51.657012 26126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "3b50a399fed04839b8e99f4cbe9ab1eb"
format_stamp: "Formatted at 2026-08-06 00:47:51 on dist-test-slave-n326"
server_key: "38503d19b626252223c458e37b76dd24"
server_key_iv: "cc02be0b13bddb8d6267a2e9f32f12e7"
server_key_version: "encryptionkey@0"
I20260806 00:47:51.660421 26126 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:47:51.662650 26146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:51.663791 26126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:47:51.663930 26126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "3b50a399fed04839b8e99f4cbe9ab1eb"
format_stamp: "Formatted at 2026-08-06 00:47:51 on dist-test-slave-n326"
server_key: "38503d19b626252223c458e37b76dd24"
server_key_iv: "cc02be0b13bddb8d6267a2e9f32f12e7"
server_key_version: "encryptionkey@0"
I20260806 00:47:51.664036 26126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:51.690481 26126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:51.693687 26126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:51.693881 26126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:51.700826 26126 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:38893
I20260806 00:47:51.700840 26233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:38893 every 8 connection(s)
I20260806 00:47:51.701682 26126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:51.704217 26235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:51.707628 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 26126
I20260806 00:47:51.707784 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:51.708082 30034 external_mini_cluster.cc:1428] Setting key 127a17339c0c0f0809ee72c9515cf70e
I20260806 00:47:51.709597 26235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb: Bootstrap starting.
I20260806 00:47:51.711963 26235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:51.712647 26235 log.cc:824] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:51.714332 26235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb: No bootstrap required, opened a new log
Aug 06 00:47:51 dist-test-slave-n326 krb5kdc[26112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:51.718256 26235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b50a399fed04839b8e99f4cbe9ab1eb" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38893 } }
I20260806 00:47:51.718454 26235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:51.718549 26235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b50a399fed04839b8e99f4cbe9ab1eb, State: Initialized, Role: FOLLOWER
I20260806 00:47:51.718973 26235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b50a399fed04839b8e99f4cbe9ab1eb" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38893 } }
I20260806 00:47:51.719156 26235 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:51.719250 26235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:51.719334 26235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:51.720130 26235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b50a399fed04839b8e99f4cbe9ab1eb" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38893 } }
I20260806 00:47:51.720459 26235 leader_election.cc:302] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b50a399fed04839b8e99f4cbe9ab1eb; no voters:
I20260806 00:47:51.720778 26235 leader_election.cc:288] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:47:51.720839 26241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:51.721098 26241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [term 1 LEADER]: Becoming Leader. State: Replica: 3b50a399fed04839b8e99f4cbe9ab1eb, State: Running, Role: LEADER
I20260806 00:47:51.721467 26241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b50a399fed04839b8e99f4cbe9ab1eb" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38893 } }
I20260806 00:47:51.722033 26235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:51.722162 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:51.709395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45160 (local address 127.29.84.190:38893)
0806 00:47:51.709769 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:51.709783 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:51.709817 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:47:51.710385 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:51.710399 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:51.710691 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:51.711123 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:51.711967 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:51.713025 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:51.713864 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:51.714336 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:51.714514 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:51.717140 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:51.717145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:51.717157 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:51.717182 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:51.719120 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:51.719599 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:51.719607 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:51.719613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:51.719735 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:51.720029 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:51.720032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:51.720034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:51.720465 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:51.720695 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:51.720986 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 00:47:51.721317 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":122,"threads_started":1}
I20260806 00:47:51.722775 26241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b50a399fed04839b8e99f4cbe9ab1eb. Latest consensus state: current_term: 1 leader_uuid: "3b50a399fed04839b8e99f4cbe9ab1eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b50a399fed04839b8e99f4cbe9ab1eb" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38893 } } }
I20260806 00:47:51.722916 26241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:51.723742 26244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b50a399fed04839b8e99f4cbe9ab1eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b50a399fed04839b8e99f4cbe9ab1eb" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 38893 } } }
I20260806 00:47:51.723850 26244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb [sys.catalog]: This master's current role is: LEADER
W20260806 00:47:51.726938 26261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:47:51.727022 26261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:47:51.727121 26255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:51.727809 26255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:51.732645 26255 catalog_manager.cc:1347] Generated new cluster ID: cacfa4ec41924ce1801288c89193a933
I20260806 00:47:51.732718 26255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:51.749409 26255 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:47:51.750537 26255 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:51.756533 26255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb: Generated new TSK 0
I20260806 00:47:51.757225 26255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.84.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
Entry for principal kudu/127.29.84.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab.
I20260806 00:47:51.793684 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38893
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:51.928363 26272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:51.928582 26272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:51.928622 26272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:51.932596 26272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:51.932644 26272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:51.932740 26272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:47:51.937639 26272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.129.keytab
--principal=kudu/127.29.84.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:51.938498 26272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:51.939436 26272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:51.948189 26284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:51.948199 26279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:51.948277 26281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:51.948362 26272 server_base.cc:1034] running on GCE node
I20260806 00:47:51.948769 26272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:51.949316 26272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:51.950501 26272 hybrid_clock.cc:648] HybridClock initialized: now 1785977271950471 us; error 46 us; skew 500 ppm
Aug 06 00:47:51 dist-test-slave-n326 krb5kdc[26112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977271, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:51.953119 26272 init.cc:377] Logged in from keytab as kudu/127.29.84.129@KRBTEST.COM (short username kudu)
I20260806 00:47:51.953899 26272 webserver.cc:460] Webserver started at http://127.29.84.129:44155/ using document root <none> and password file <none>
I20260806 00:47:51.954393 26272 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:51.954484 26272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:51.954702 26272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:51.956632 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "1457d635f7614803ac5bf79d62082551"
format_stamp: "Formatted at 2026-08-06 00:47:51 on dist-test-slave-n326"
server_key: "c1a9a98acf298ae3db21705806806034"
server_key_iv: "004148a76990df4a43d936dd3585f40b"
server_key_version: "encryptionkey@0"
I20260806 00:47:51.957173 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "1457d635f7614803ac5bf79d62082551"
format_stamp: "Formatted at 2026-08-06 00:47:51 on dist-test-slave-n326"
server_key: "c1a9a98acf298ae3db21705806806034"
server_key_iv: "004148a76990df4a43d936dd3585f40b"
server_key_version: "encryptionkey@0"
I20260806 00:47:51.960616 26272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:47:51.962770 26296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:51.963899 26272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:51.964032 26272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "1457d635f7614803ac5bf79d62082551"
format_stamp: "Formatted at 2026-08-06 00:47:51 on dist-test-slave-n326"
server_key: "c1a9a98acf298ae3db21705806806034"
server_key_iv: "004148a76990df4a43d936dd3585f40b"
server_key_version: "encryptionkey@0"
I20260806 00:47:51.964138 26272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:51.977036 26272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:51.979712 26272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:51.979894 26272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:51.980417 26272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:51.981148 26272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:51.981215 26272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:51.981276 26272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:51.981325 26272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:51.989913 26272 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:34997
I20260806 00:47:51.989954 26495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:34997 every 8 connection(s)
I20260806 00:47:51.990700 26272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
Aug 06 00:47:51 dist-test-slave-n326 krb5kdc[26112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977271, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.129@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:52.000042 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 26272
I20260806 00:47:52.000198 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:47:52.000536 30034 external_mini_cluster.cc:1428] Setting key eb8383a0e503a0c9f10b5a722caa4a1e
I20260806 00:47:52.003739 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:51.992299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:58671 (local address 127.29.84.190:38893)
0806 00:47:51.992438 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:51.992443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:51.993230 (+ 787us) server_negotiation.cc:407] Connection header received
0806 00:47:51.994381 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:51.994384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:51.994428 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:51.994501 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:51.995598 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:51.996149 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:51.997249 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:51.997476 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:51.997580 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:51.999946 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:51.999949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:51.999951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:51.999979 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:52.001677 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:52.002252 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:52.002255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:52.002256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:52.002328 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:52.002699 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:52.002701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:52.002702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:52.002852 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:52.002923 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.003534 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.003629 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:47:52.004637 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:51.992684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38893 (local address 127.29.84.129:58671)
0806 00:47:51.993088 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:51.993126 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:51.994203 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:51.994653 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:51.994659 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:51.994952 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:51.995474 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:51.995487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:51.996256 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:51.996258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:51.997127 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:51.997136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:51.997613 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:51.997615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:51.997740 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:51.998267 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:51.998283 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:51.999771 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:52.001791 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:52.001798 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:52.001811 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:52.002142 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:52.002421 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:52.002426 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:52.002431 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:52.002607 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:52.002959 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:52.002968 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:52.003283 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 00:47:52.003478 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 00:47:52.003801 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":111,"threads_started":1}
I20260806 00:47:52.005884 26496 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38893
I20260806 00:47:52.006105 26496 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:52.006534 26496 heartbeater.cc:511] Master 127.29.84.190:38893 requested a full tablet report, sending...
I20260806 00:47:52.008211 26180 ts_manager.cc:194] Registered new tserver with Master: 1457d635f7614803ac5bf79d62082551 (127.29.84.129:34997)
I20260806 00:47:52.009835 26180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.129@KRBTEST.COM'} at 127.29.84.129:58671
WARNING: no policy specified for kudu/127.29.84.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
Entry for principal kudu/127.29.84.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab.
I20260806 00:47:52.027982 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38893
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:52.132752 26507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:52.133002 26507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:52.133101 26507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:52.136963 26507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:52.137044 26507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:52.137135 26507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:47:52.142066 26507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.130.keytab
--principal=kudu/127.29.84.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:52.142979 26507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:52.143896 26507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:52.152879 26513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:52.152884 26518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:52.152922 26507 server_base.cc:1034] running on GCE node
W20260806 00:47:52.152878 26514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:52.153522 26507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:52.154055 26507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:52.155236 26507 hybrid_clock.cc:648] HybridClock initialized: now 1785977272155208 us; error 45 us; skew 500 ppm
Aug 06 00:47:52 dist-test-slave-n326 krb5kdc[26112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977272, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:52.157794 26507 init.cc:377] Logged in from keytab as kudu/127.29.84.130@KRBTEST.COM (short username kudu)
I20260806 00:47:52.158596 26507 webserver.cc:460] Webserver started at http://127.29.84.130:46557/ using document root <none> and password file <none>
I20260806 00:47:52.159193 26507 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:52.159287 26507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:52.159510 26507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:52.161396 26507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "6180839057c54ec9b523ae395cfa903d"
format_stamp: "Formatted at 2026-08-06 00:47:52 on dist-test-slave-n326"
server_key: "034bc72b2ed8cc0985cad1c94011fa2f"
server_key_iv: "af4c8ead0191ced421c5df64094b114f"
server_key_version: "encryptionkey@0"
I20260806 00:47:52.161984 26507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "6180839057c54ec9b523ae395cfa903d"
format_stamp: "Formatted at 2026-08-06 00:47:52 on dist-test-slave-n326"
server_key: "034bc72b2ed8cc0985cad1c94011fa2f"
server_key_iv: "af4c8ead0191ced421c5df64094b114f"
server_key_version: "encryptionkey@0"
I20260806 00:47:52.165557 26507 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:47:52.167893 26532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:52.169111 26507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:52.169242 26507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "6180839057c54ec9b523ae395cfa903d"
format_stamp: "Formatted at 2026-08-06 00:47:52 on dist-test-slave-n326"
server_key: "034bc72b2ed8cc0985cad1c94011fa2f"
server_key_iv: "af4c8ead0191ced421c5df64094b114f"
server_key_version: "encryptionkey@0"
I20260806 00:47:52.169348 26507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:52.178472 26507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:52.180950 26507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:52.181145 26507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:52.181691 26507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:52.182441 26507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:52.182511 26507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:52.182585 26507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:52.182627 26507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:52.190860 26507 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:34615
I20260806 00:47:52.190883 26705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:34615 every 8 connection(s)
I20260806 00:47:52.191676 26507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:47:52.193112 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 26507
I20260806 00:47:52.193257 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:47:52.193573 30034 external_mini_cluster.cc:1428] Setting key 2961ed0104f2e623afe0fbe36a3bd005
Aug 06 00:47:52 dist-test-slave-n326 krb5kdc[26112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977272, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.130@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.84.131@KRBTEST.COM; defaulting to no policy
I20260806 00:47:52.206646 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.193364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:51373 (local address 127.29.84.190:38893)
0806 00:47:52.193535 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.193539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.194417 (+ 878us) server_negotiation.cc:407] Connection header received
0806 00:47:52.195913 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.195919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.195980 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.196078 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:52.197152 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.197699 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.199101 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.199437 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.199557 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.202808 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:52.202813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:52.202819 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:52.202852 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:52.204493 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:52.204961 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:52.204964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:52.204965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:52.205047 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:52.205388 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:52.205393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:52.205395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:52.205630 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:52.205725 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.206368 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.206483 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:47:52.207617 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.193775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38893 (local address 127.29.84.130:51373)
0806 00:47:52.194258 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:52.194298 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:52.195689 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:52.196216 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:52.196223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:52.196532 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:52.196971 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.196993 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.197829 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.197834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.198941 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.198951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.199645 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.199654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.199895 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.200549 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:52.200575 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:52.202611 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:52.204611 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:52.204615 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:52.204624 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:52.204853 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:52.205166 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:52.205169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:52.205171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:52.205290 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:52.205786 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:52.205792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:52.206054 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 00:47:52.206288 (+ 234us) client_negotiation.cc:240] Negotiation successful
0806 00:47:52.206591 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":141,"threads_started":1}
I20260806 00:47:52.209002 26706 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38893
I20260806 00:47:52.209247 26706 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.131@KRBTEST.COM" created.
I20260806 00:47:52.209834 26706 heartbeater.cc:511] Master 127.29.84.190:38893 requested a full tablet report, sending...
I20260806 00:47:52.210855 26180 ts_manager.cc:194] Registered new tserver with Master: 6180839057c54ec9b523ae395cfa903d (127.29.84.130:34615)
I20260806 00:47:52.211680 26180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.130@KRBTEST.COM'} at 127.29.84.130:51373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
Entry for principal kudu/127.29.84.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab.
I20260806 00:47:52.226361 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38893
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:52.332342 26714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:52.332559 26714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:52.332595 26714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:52.336417 26714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:52.336467 26714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:52.336565 26714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:47:52.341555 26714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.131.keytab
--principal=kudu/127.29.84.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:38893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:52.342420 26714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:52.343344 26714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:52.352288 26722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:52.352342 26723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:52.352396 26714 server_base.cc:1034] running on GCE node
W20260806 00:47:52.352658 26726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:52.353062 26714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:52.353606 26714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:52.354799 26714 hybrid_clock.cc:648] HybridClock initialized: now 1785977272354783 us; error 31 us; skew 500 ppm
Aug 06 00:47:52 dist-test-slave-n326 krb5kdc[26112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977272, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:52.357580 26714 init.cc:377] Logged in from keytab as kudu/127.29.84.131@KRBTEST.COM (short username kudu)
I20260806 00:47:52.358433 26714 webserver.cc:460] Webserver started at http://127.29.84.131:46551/ using document root <none> and password file <none>
I20260806 00:47:52.359016 26714 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:52.359135 26714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:52.359362 26714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:52.361266 26714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "29e1739b396f493682ab42cb915098f0"
format_stamp: "Formatted at 2026-08-06 00:47:52 on dist-test-slave-n326"
server_key: "757acfd2c8f72340b194c1eebd9101c0"
server_key_iv: "87b84847ff3eb243cdf821365e6debf1"
server_key_version: "encryptionkey@0"
I20260806 00:47:52.361841 26714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "29e1739b396f493682ab42cb915098f0"
format_stamp: "Formatted at 2026-08-06 00:47:52 on dist-test-slave-n326"
server_key: "757acfd2c8f72340b194c1eebd9101c0"
server_key_iv: "87b84847ff3eb243cdf821365e6debf1"
server_key_version: "encryptionkey@0"
I20260806 00:47:52.365384 26714 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 00:47:52.367821 26738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:52.369057 26714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:52.369184 26714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "29e1739b396f493682ab42cb915098f0"
format_stamp: "Formatted at 2026-08-06 00:47:52 on dist-test-slave-n326"
server_key: "757acfd2c8f72340b194c1eebd9101c0"
server_key_iv: "87b84847ff3eb243cdf821365e6debf1"
server_key_version: "encryptionkey@0"
I20260806 00:47:52.369287 26714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:52.387185 26714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:52.389839 26714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:52.390044 26714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:52.390578 26714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:52.391384 26714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:52.391463 26714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:52.391531 26714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:52.391573 26714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:52.400256 26714 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:43897
I20260806 00:47:52.400286 26910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:43897 every 8 connection(s)
I20260806 00:47:52.401051 26714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:47:52.402419 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 26714
I20260806 00:47:52.402565 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:47:52.402904 30034 external_mini_cluster.cc:1428] Setting key 5f50e5f8e2dd096a9bbeebc497bb2bea
Aug 06 00:47:52 dist-test-slave-n326 krb5kdc[26112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977272, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.131@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:47:52.413715 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.402831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:59441 (local address 127.29.84.190:38893)
0806 00:47:52.402997 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.403002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.403935 (+ 933us) server_negotiation.cc:407] Connection header received
0806 00:47:52.405509 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.405511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.405555 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.405620 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:52.406660 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.407275 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.408337 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.408555 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.408632 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.410778 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:52.410781 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:52.410783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:52.410804 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:52.412023 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:52.412477 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:52.412479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:52.412480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:52.412556 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:52.412841 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:52.412843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:52.412844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:52.413015 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:52.413104 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.413492 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.413603 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 00:47:52.414425 26919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.403274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:38893 (local address 127.29.84.131:59441)
0806 00:47:52.403785 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:52.403854 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:52.405334 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:52.405765 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:52.405771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:52.406086 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:52.406534 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.406547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.407382 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.407385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.408230 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.408239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.408677 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.408679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.408815 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.409293 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:47:52.409308 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:52.410629 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:52.412122 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:52.412126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:52.412135 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:52.412381 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:52.412638 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:47:52.412640 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:47:52.412642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:47:52.412761 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:47:52.413116 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:52.413120 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:52.413307 (+ 187us) client_negotiation.cc:769] Sending connection context
0806 00:47:52.413443 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 00:47:52.413686 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":150,"threads_started":1}
I20260806 00:47:52.415719 26914 heartbeater.cc:347] Connected to a master server at 127.29.84.190:38893
I20260806 00:47:52.415949 26914 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:52.416435 26914 heartbeater.cc:511] Master 127.29.84.190:38893 requested a full tablet report, sending...
I20260806 00:47:52.417464 26180 ts_manager.cc:194] Registered new tserver with Master: 29e1739b396f493682ab42cb915098f0 (127.29.84.131:43897)
I20260806 00:47:52.418022 26180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.84.131@KRBTEST.COM'} at 127.29.84.131:59441
I20260806 00:47:52.428351 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:47:52.436492 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.430268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.29.84.190:38893)
0806 00:47:52.430375 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.430378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.430522 (+ 144us) server_negotiation.cc:407] Connection header received
0806 00:47:52.430592 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.430594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.430626 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.430688 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:52.431324 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.431751 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.432472 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.432649 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.432716 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.433856 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:52.433858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:52.433860 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:52.433880 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:52.434994 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:52.435391 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:52.435393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:52.435394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:52.435481 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:52.435752 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:52.435755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:52.435756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:52.435989 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:52.436108 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.436280 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.436372 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:47:52.439213 26180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:47:52.442137 26180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:47:52.456199 26933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.450232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:43897 (local address 127.0.0.1:43650)
0806 00:47:52.450863 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:52.451346 (+ 483us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:52.451432 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:52.451918 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:52.451924 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:52.451944 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:52.452080 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.452086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.453273 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.453278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.454468 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.454480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.455679 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.455682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.455785 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.455828 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 00:47:52.455949 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 00:47:52.456062 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260806 00:47:52.456221 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.450757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43650 (local address 127.29.84.131:43897)
0806 00:47:52.451208 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.451221 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.451537 (+ 316us) server_negotiation.cc:407] Connection header received
0806 00:47:52.451610 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.451615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.451765 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.451906 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:52.452191 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.453152 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.454600 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.455554 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.455668 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.455809 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.455990 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.456083 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":159,"threads_started":1}
I20260806 00:47:52.456360 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.450040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:34997 (local address 127.0.0.1:38066)
0806 00:47:52.450956 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:52.450973 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:52.451126 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:52.452539 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:52.452541 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:52.452560 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:52.452636 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.452641 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.454082 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.454086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.455107 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.455117 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.456012 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.456015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.456092 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.456112 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:47:52.456159 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 00:47:52.456219 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":46,"thread_start_us":86,"threads_started":1}
I20260806 00:47:52.457093 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.450554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:34615 (local address 127.0.0.1:40476)
0806 00:47:52.451650 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:52.451661 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:52.451774 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:52.452261 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:52.452264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:52.452283 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:52.452354 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.452360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.453937 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.453942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.455390 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.455403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.456707 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.456711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.456815 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.456835 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:47:52.456909 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:47:52.456969 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"mutex_wait_us":67,"thread_start_us":93,"threads_started":1}
I20260806 00:47:52.457101 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.450253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38066 (local address 127.29.84.129:34997)
0806 00:47:52.452051 (+ 1798us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.452063 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.452085 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:47:52.452161 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.452166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.452338 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.452551 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:52.452742 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.453937 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.455228 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.455873 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.456687 (+ 814us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.456827 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.456877 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.456975 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1681,"thread_start_us":182,"threads_started":1}
I20260806 00:47:52.457726 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.451013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40476 (local address 127.29.84.130:34615)
0806 00:47:52.451525 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.451537 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.451754 (+ 217us) server_negotiation.cc:407] Connection header received
0806 00:47:52.451915 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.451920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.452131 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.452280 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:52.452455 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.453731 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.455551 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.456441 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.456561 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.456705 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.457504 (+ 799us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.457596 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":176,"threads_started":1}
I20260806 00:47:52.459506 26809 tablet_service.cc:1467] Processing CreateTablet for tablet f31d5a0e5d2c408c904e14d6c0db538d (DEFAULT_TABLE table=test-table [id=52a2ec5532fd4662b1274f41e39b27d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:52.459925 26369 tablet_service.cc:1467] Processing CreateTablet for tablet f31d5a0e5d2c408c904e14d6c0db538d (DEFAULT_TABLE table=test-table [id=52a2ec5532fd4662b1274f41e39b27d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:52.460647 26809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f31d5a0e5d2c408c904e14d6c0db538d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:52.460788 26369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f31d5a0e5d2c408c904e14d6c0db538d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:52.460404 26603 tablet_service.cc:1467] Processing CreateTablet for tablet f31d5a0e5d2c408c904e14d6c0db538d (DEFAULT_TABLE table=test-table [id=52a2ec5532fd4662b1274f41e39b27d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:52.461414 26603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f31d5a0e5d2c408c904e14d6c0db538d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:52.466475 26941 tablet_bootstrap.cc:492] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551: Bootstrap starting.
I20260806 00:47:52.469049 26941 tablet_bootstrap.cc:654] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:52.469353 26943 tablet_bootstrap.cc:492] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: Bootstrap starting.
I20260806 00:47:52.469440 26942 tablet_bootstrap.cc:492] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: Bootstrap starting.
I20260806 00:47:52.470542 26941 log.cc:824] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:52.471516 26943 tablet_bootstrap.cc:654] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:52.471611 26942 tablet_bootstrap.cc:654] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:52.472724 26943 log.cc:824] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:52.472930 26941 tablet_bootstrap.cc:492] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551: No bootstrap required, opened a new log
I20260806 00:47:52.472954 26942 log.cc:824] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:52.473127 26941 ts_tablet_manager.cc:1397] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 00:47:52.474826 26942 tablet_bootstrap.cc:492] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: No bootstrap required, opened a new log
I20260806 00:47:52.475042 26942 ts_tablet_manager.cc:1397] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:47:52.476452 26943 tablet_bootstrap.cc:492] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: No bootstrap required, opened a new log
I20260806 00:47:52.476408 26941 raft_consensus.cc:348] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } }
I20260806 00:47:52.476627 26943 ts_tablet_manager.cc:1397] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 00:47:52.476677 26941 raft_consensus.cc:374] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:52.476733 26941 raft_consensus.cc:729] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1457d635f7614803ac5bf79d62082551, State: Initialized, Role: FOLLOWER
I20260806 00:47:52.477298 26941 consensus_queue.cc:260] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } }
I20260806 00:47:52.477968 26496 heartbeater.cc:503] Master 127.29.84.190:38893 was elected leader, sending a full tablet report...
I20260806 00:47:52.478267 26941 ts_tablet_manager.cc:1428] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:47:52.479051 26942 raft_consensus.cc:348] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } }
I20260806 00:47:52.479327 26942 raft_consensus.cc:374] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:52.479387 26942 raft_consensus.cc:729] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29e1739b396f493682ab42cb915098f0, State: Initialized, Role: FOLLOWER
I20260806 00:47:52.479784 26942 consensus_queue.cc:260] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } }
I20260806 00:47:52.480597 26914 heartbeater.cc:503] Master 127.29.84.190:38893 was elected leader, sending a full tablet report...
I20260806 00:47:52.480315 26943 raft_consensus.cc:348] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } }
I20260806 00:47:52.480997 26943 raft_consensus.cc:374] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:52.481069 26943 raft_consensus.cc:729] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6180839057c54ec9b523ae395cfa903d, State: Initialized, Role: FOLLOWER
I20260806 00:47:52.481489 26943 consensus_queue.cc:260] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } }
I20260806 00:47:52.481693 26942 ts_tablet_manager.cc:1428] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:47:52.482111 26706 heartbeater.cc:503] Master 127.29.84.190:38893 was elected leader, sending a full tablet report...
I20260806 00:47:52.482455 26943 ts_tablet_manager.cc:1428] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 00:47:52.491740 26499 tablet.cc:2369] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:47:52.516470 26948 raft_consensus.cc:482] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:52.516597 26948 raft_consensus.cc:504] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } }
I20260806 00:47:52.517503 26948 leader_election.cc:288] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29e1739b396f493682ab42cb915098f0 (127.29.84.131:43897), 6180839057c54ec9b523ae395cfa903d (127.29.84.130:34615)
I20260806 00:47:52.521287 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.517763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:32869 (local address 127.29.84.131:43897)
0806 00:47:52.517886 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.517908 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.518072 (+ 164us) server_negotiation.cc:407] Connection header received
0806 00:47:52.518260 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.518264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.518316 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.518372 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:52.518689 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.519243 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.520324 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.520717 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.520833 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.520903 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.521089 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.521156 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:47:52.521294 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.517792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:53909 (local address 127.29.84.130:34615)
0806 00:47:52.517907 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.517911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.518148 (+ 237us) server_negotiation.cc:407] Connection header received
0806 00:47:52.518290 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.518292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.518331 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.518423 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:52.518761 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.519275 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.520365 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.520727 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.520873 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.520931 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.521116 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.521195 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:47:52.521307 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.517718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:34615 (local address 127.29.84.129:53909)
0806 00:47:52.518072 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:52.518083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:52.518164 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:52.518490 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:52.518495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:52.518533 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:52.518656 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.518662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.519403 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.519406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.520225 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.520234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.520878 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.520881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.520982 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.520999 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 00:47:52.521089 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 00:47:52.521159 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":53,"thread_start_us":108,"threads_started":1}
I20260806 00:47:52.521314 26952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.517677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:43897 (local address 127.29.84.129:32869)
0806 00:47:52.517984 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:52.517996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:52.518112 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:52.518416 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:52.518426 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:52.518443 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:52.518577 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.518584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.519345 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.519350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.520204 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:52.520213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.520835 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.520839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:52.520957 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.520980 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:47:52.521048 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 00:47:52.521138 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":7,"thread_start_us":81,"threads_started":1}
I20260806 00:47:52.522025 26635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31d5a0e5d2c408c904e14d6c0db538d" candidate_uuid: "1457d635f7614803ac5bf79d62082551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6180839057c54ec9b523ae395cfa903d" is_pre_election: true
I20260806 00:47:52.522193 26837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31d5a0e5d2c408c904e14d6c0db538d" candidate_uuid: "1457d635f7614803ac5bf79d62082551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29e1739b396f493682ab42cb915098f0" is_pre_election: true
I20260806 00:47:52.522327 26635 raft_consensus.cc:2454] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1457d635f7614803ac5bf79d62082551 in term 0.
I20260806 00:47:52.522487 26837 raft_consensus.cc:2454] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1457d635f7614803ac5bf79d62082551 in term 0.
I20260806 00:47:52.522830 26301 leader_election.cc:302] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1457d635f7614803ac5bf79d62082551, 6180839057c54ec9b523ae395cfa903d; no voters:
I20260806 00:47:52.523200 26948 raft_consensus.cc:2781] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:47:52.523288 26948 raft_consensus.cc:482] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:47:52.523339 26948 raft_consensus.cc:3037] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:52.524421 26948 raft_consensus.cc:504] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } }
I20260806 00:47:52.524761 26948 leader_election.cc:288] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [CANDIDATE]: Term 1 election: Requested vote from peers 29e1739b396f493682ab42cb915098f0 (127.29.84.131:43897), 6180839057c54ec9b523ae395cfa903d (127.29.84.130:34615)
I20260806 00:47:52.525110 26837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31d5a0e5d2c408c904e14d6c0db538d" candidate_uuid: "1457d635f7614803ac5bf79d62082551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29e1739b396f493682ab42cb915098f0"
I20260806 00:47:52.525213 26635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31d5a0e5d2c408c904e14d6c0db538d" candidate_uuid: "1457d635f7614803ac5bf79d62082551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6180839057c54ec9b523ae395cfa903d"
I20260806 00:47:52.525255 26837 raft_consensus.cc:3037] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:52.525321 26635 raft_consensus.cc:3037] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:52.526515 26635 raft_consensus.cc:2454] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1457d635f7614803ac5bf79d62082551 in term 1.
I20260806 00:47:52.526515 26837 raft_consensus.cc:2454] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1457d635f7614803ac5bf79d62082551 in term 1.
I20260806 00:47:52.526875 26301 leader_election.cc:302] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1457d635f7614803ac5bf79d62082551, 6180839057c54ec9b523ae395cfa903d; no voters:
I20260806 00:47:52.527066 26948 raft_consensus.cc:2781] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:52.527283 26948 raft_consensus.cc:686] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 1 LEADER]: Becoming Leader. State: Replica: 1457d635f7614803ac5bf79d62082551, State: Running, Role: LEADER
I20260806 00:47:52.527631 26948 consensus_queue.cc:237] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } }
I20260806 00:47:52.536232 26180 catalog_manager.cc:5495] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1457d635f7614803ac5bf79d62082551 (127.29.84.129). New cstate: current_term: 1 leader_uuid: "1457d635f7614803ac5bf79d62082551" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1457d635f7614803ac5bf79d62082551" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 34997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 } health_report { overall_health: UNKNOWN } } }
I20260806 00:47:52.558394 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.554073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38068 (local address 127.29.84.129:34997)
0806 00:47:52.554210 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.554213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.554245 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:47:52.554426 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.554429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.554495 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.554609 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:47:52.555029 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.555615 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.556796 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.557169 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.557279 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.557633 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:47:52.557997 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:47:52.558096 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.558223 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.558269 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:47:52.566709 26837 raft_consensus.cc:1261] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [term 1 FOLLOWER]: Refusing update from remote peer 1457d635f7614803ac5bf79d62082551: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:52.566769 26635 raft_consensus.cc:1261] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [term 1 FOLLOWER]: Refusing update from remote peer 1457d635f7614803ac5bf79d62082551: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:52.567556 26954 consensus_queue.cc:1035] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29e1739b396f493682ab42cb915098f0" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 43897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:52.567747 26948 consensus_queue.cc:1035] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6180839057c54ec9b523ae395cfa903d" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 34615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:52.576798 26957 mvcc.cc:205] Tried to move back new op lower bound from 7315362908426096640 to 7315362908275896320. Current Snapshot: MvccSnapshot[applied={T|T < 7315362908426096640}]
I20260806 00:47:52.578828 26958 mvcc.cc:205] Tried to move back new op lower bound from 7315362908426096640 to 7315362908275896320. Current Snapshot: MvccSnapshot[applied={T|T < 7315362908426096640 or (T in {7315362908426096640})}]
I20260806 00:47:52.585078 26180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:52.585258 26180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:52.588271 26180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b50a399fed04839b8e99f4cbe9ab1eb: Sending DeleteTablet for 3 replicas of tablet f31d5a0e5d2c408c904e14d6c0db538d
I20260806 00:47:52.589075 26809 tablet_service.cc:1514] Processing DeleteTablet for tablet f31d5a0e5d2c408c904e14d6c0db538d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:52 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:43650
I20260806 00:47:52.589239 26369 tablet_service.cc:1514] Processing DeleteTablet for tablet f31d5a0e5d2c408c904e14d6c0db538d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:52 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:38066
I20260806 00:47:52.589318 26603 tablet_service.cc:1514] Processing DeleteTablet for tablet f31d5a0e5d2c408c904e14d6c0db538d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:52 UTC) from {username='kudu', principal='kudu/127.29.84.190@KRBTEST.COM'} at 127.0.0.1:40476
I20260806 00:47:52.589712 26969 tablet_replica.cc:330] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551: stopping tablet replica
I20260806 00:47:52.590093 26969 raft_consensus.cc:2229] T f31d5a0e5d2c408c904e14d6c0db538d P 1457d635f7614803ac5bf79d62082551 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:47:52.590410 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 26272
I20260806 00:47:52.592749 26968 tablet_replica.cc:330] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: stopping tablet replica
I20260806 00:47:52.593060 26968 raft_consensus.cc:2229] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:52.593379 26968 raft_consensus.cc:2258] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:52.594345 26972 tablet_replica.cc:330] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: stopping tablet replica
I20260806 00:47:52.594673 26972 raft_consensus.cc:2229] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:52.594998 26972 raft_consensus.cc:2258] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:52.598968 26968 ts_tablet_manager.cc:1905] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:52.598968 26972 ts_tablet_manager.cc:1905] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:52.601601 26972 ts_tablet_manager.cc:1918] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:52.601601 26968 ts_tablet_manager.cc:1918] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:52.601706 26968 log.cc:1196] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/f31d5a0e5d2c408c904e14d6c0db538d
I20260806 00:47:52.601706 26972 log.cc:1196] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/f31d5a0e5d2c408c904e14d6c0db538d
I20260806 00:47:52.602066 26968 ts_tablet_manager.cc:1939] T f31d5a0e5d2c408c904e14d6c0db538d P 29e1739b396f493682ab42cb915098f0: Deleting consensus metadata
I20260806 00:47:52.602082 26972 ts_tablet_manager.cc:1939] T f31d5a0e5d2c408c904e14d6c0db538d P 6180839057c54ec9b523ae395cfa903d: Deleting consensus metadata
I20260806 00:47:52.603317 26154 catalog_manager.cc:4841] TS 6180839057c54ec9b523ae395cfa903d (127.29.84.130:34615): tablet f31d5a0e5d2c408c904e14d6c0db538d (table test-table [id=52a2ec5532fd4662b1274f41e39b27d9]) successfully deleted
I20260806 00:47:52.603519 26155 catalog_manager.cc:4841] TS 29e1739b396f493682ab42cb915098f0 (127.29.84.131:43897): tablet f31d5a0e5d2c408c904e14d6c0db538d (table test-table [id=52a2ec5532fd4662b1274f41e39b27d9]) successfully deleted
W20260806 00:47:52.603794 26150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:34997 (error 108)
I20260806 00:47:52.603945 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 26507
I20260806 00:47:52.613190 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.612876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:34997 (local address 127.0.0.1:38082)
0806 00:47:52.613022 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:52.613096 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:34997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 00:47:52.613341 26150 catalog_manager.cc:4584] TS 1457d635f7614803ac5bf79d62082551 (127.29.84.129:34997): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f31d5a0e5d2c408c904e14d6c0db538d: Network error: Client connection negotiation failed: client connection to 127.29.84.129:34997: connect: Connection refused (error 111)
I20260806 00:47:52.624009 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 26714
I20260806 00:47:52.631871 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 26126
2026-08-06T00:47:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2439 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T00:47:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:52Z Disabled control of system clock
I20260806 00:47:52.666860 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:42275
--webserver_interface=127.29.84.190
--webserver_port=0
--builtin_ntp_servers=127.29.84.148:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:42275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:47:52.771991 26979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:52.772202 26979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:52.772233 26979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:52.775844 26979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:52.775888 26979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:52.775902 26979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:52.775918 26979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:52.775962 26979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:52.780360 26979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:42275
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:42275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:52.781215 26979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:52.782501 26979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:52.790880 26992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:52.790884 26991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:52.790823 26996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:52.791098 26979 server_base.cc:1034] running on GCE node
I20260806 00:47:52.791711 26979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:52.792709 26979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:52.793931 26979 hybrid_clock.cc:648] HybridClock initialized: now 1785977272793908 us; error 35 us; skew 500 ppm
I20260806 00:47:52.795748 26979 webserver.cc:460] Webserver started at http://127.29.84.190:33983/ using document root <none> and password file <none>
I20260806 00:47:52.796355 26979 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:52.796455 26979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:52.796690 26979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:52.798784 26979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "b0745783ead24ce192b5482a2ff1a62f"
format_stamp: "Formatted at 2026-08-06 00:47:52 on dist-test-slave-n326"
server_key: "ae7aa5ab9d24bae46ca6b4af4bfc929c"
server_key_iv: "1931d7da66672f89c115da60f985eb5d"
server_key_version: "encryptionkey@0"
I20260806 00:47:52.799379 26979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "b0745783ead24ce192b5482a2ff1a62f"
format_stamp: "Formatted at 2026-08-06 00:47:52 on dist-test-slave-n326"
server_key: "ae7aa5ab9d24bae46ca6b4af4bfc929c"
server_key_iv: "1931d7da66672f89c115da60f985eb5d"
server_key_version: "encryptionkey@0"
I20260806 00:47:52.803041 26979 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:47:52.805414 27004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:52.806567 26979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:52.806716 26979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "b0745783ead24ce192b5482a2ff1a62f"
format_stamp: "Formatted at 2026-08-06 00:47:52 on dist-test-slave-n326"
server_key: "ae7aa5ab9d24bae46ca6b4af4bfc929c"
server_key_iv: "1931d7da66672f89c115da60f985eb5d"
server_key_version: "encryptionkey@0"
I20260806 00:47:52.806826 26979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:52.837236 26979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:52.837914 26979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:52.838128 26979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:52.845202 26979 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:42275
I20260806 00:47:52.845207 27094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:42275 every 8 connection(s)
I20260806 00:47:52.846158 26979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:52.849047 27095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:52.852383 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 26979
I20260806 00:47:52.852515 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:52.852846 30034 external_mini_cluster.cc:1428] Setting key 84508f81b70e90ce468c9e8561d6b8b6
I20260806 00:47:52.854825 27095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f: Bootstrap starting.
I20260806 00:47:52.856865 27095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:52.857589 27095 log.cc:824] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:52.859437 27095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f: No bootstrap required, opened a new log
I20260806 00:47:52.862727 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:52.854310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.29.84.190:42275)
0806 00:47:52.854982 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:52.855000 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:52.855037 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:47:52.855737 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:52.855753 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:52.856143 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:52.856496 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:52.856978 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.858287 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.859308 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:52.859861 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:52.860046 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:52.860950 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:52.860961 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:52.860973 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:52.861357 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:52.861454 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:52.861476 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:52.861592 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:52.861757 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 00:47:52.862024 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":126,"threads_started":1}
I20260806 00:47:52.863176 27095 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0745783ead24ce192b5482a2ff1a62f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42275 } }
I20260806 00:47:52.863397 27095 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:52.863472 27095 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0745783ead24ce192b5482a2ff1a62f, State: Initialized, Role: FOLLOWER
I20260806 00:47:52.863893 27095 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0745783ead24ce192b5482a2ff1a62f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42275 } }
I20260806 00:47:52.864050 27095 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:52.864117 27095 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:52.864208 27095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:52.865190 27095 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0745783ead24ce192b5482a2ff1a62f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42275 } }
I20260806 00:47:52.865458 27095 leader_election.cc:302] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0745783ead24ce192b5482a2ff1a62f; no voters:
I20260806 00:47:52.865681 27095 leader_election.cc:288] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:47:52.865790 27104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:52.866010 27104 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [term 1 LEADER]: Becoming Leader. State: Replica: b0745783ead24ce192b5482a2ff1a62f, State: Running, Role: LEADER
I20260806 00:47:52.866305 27104 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0745783ead24ce192b5482a2ff1a62f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42275 } }
I20260806 00:47:52.866772 27095 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:52.868080 27110 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0745783ead24ce192b5482a2ff1a62f. Latest consensus state: current_term: 1 leader_uuid: "b0745783ead24ce192b5482a2ff1a62f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0745783ead24ce192b5482a2ff1a62f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42275 } } }
I20260806 00:47:52.868120 27109 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0745783ead24ce192b5482a2ff1a62f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0745783ead24ce192b5482a2ff1a62f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 42275 } } }
I20260806 00:47:52.868211 27110 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:52.868216 27109 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:52.868712 27121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:52.871687 27121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:52.876291 27121 catalog_manager.cc:1347] Generated new cluster ID: 8590605e8d2d4740906ff89d548b944e
I20260806 00:47:52.876349 27121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:52.896759 27121 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:47:52.897954 27121 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:52.917779 27121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f: Generated new TSK 0
I20260806 00:47:52.918428 27121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:47:52.925782 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.129:0
--local_ip_for_outbound_sockets=127.29.84.129
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42275
--builtin_ntp_servers=127.29.84.148:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:47:53.037681 27133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:53.037941 27133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:53.038015 27133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:53.041868 27133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:53.041944 27133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:53.042030 27133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.129
I20260806 00:47:53.046448 27133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.84.129
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.129
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:53.047399 27133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:53.048635 27133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:53.057965 27144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:53.057937 27148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:53.057982 27143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:53.058173 27133 server_base.cc:1034] running on GCE node
I20260806 00:47:53.058614 27133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:53.059252 27133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:53.060407 27133 hybrid_clock.cc:648] HybridClock initialized: now 1785977273060397 us; error 24 us; skew 500 ppm
I20260806 00:47:53.067492 27133 webserver.cc:460] Webserver started at http://127.29.84.129:46791/ using document root <none> and password file <none>
I20260806 00:47:53.068120 27133 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:53.068188 27133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:53.068394 27133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:53.070375 27133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/data/instance:
uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac"
format_stamp: "Formatted at 2026-08-06 00:47:53 on dist-test-slave-n326"
server_key: "36a5b26d5725c014502b51e07da86747"
server_key_iv: "c4d1022835f349be6ad550d411fa2309"
server_key_version: "encryptionkey@0"
I20260806 00:47:53.070904 27133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance:
uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac"
format_stamp: "Formatted at 2026-08-06 00:47:53 on dist-test-slave-n326"
server_key: "36a5b26d5725c014502b51e07da86747"
server_key_iv: "c4d1022835f349be6ad550d411fa2309"
server_key_version: "encryptionkey@0"
I20260806 00:47:53.074194 27133 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:47:53.076424 27162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:53.077538 27133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:47:53.077672 27133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/wal
uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac"
format_stamp: "Formatted at 2026-08-06 00:47:53 on dist-test-slave-n326"
server_key: "36a5b26d5725c014502b51e07da86747"
server_key_iv: "c4d1022835f349be6ad550d411fa2309"
server_key_version: "encryptionkey@0"
I20260806 00:47:53.077777 27133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:53.100049 27133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:53.100601 27133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:53.100777 27133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:53.101306 27133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:53.102156 27133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:53.102227 27133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:53.102298 27133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:53.102342 27133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:53.110746 27133 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.129:36637
I20260806 00:47:53.110807 27346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.129:36637 every 8 connection(s)
I20260806 00:47:53.111605 27133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/data/info.pb
I20260806 00:47:53.112632 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 27133
I20260806 00:47:53.112771 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-0/wal/instance
I20260806 00:47:53.113082 30034 external_mini_cluster.cc:1428] Setting key 1c8f98477d0fea3e7a017bca57824d6d
I20260806 00:47:53.115270 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.130:0
--local_ip_for_outbound_sockets=127.29.84.130
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42275
--builtin_ntp_servers=127.29.84.148:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:47:53.122344 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.113329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.129:35537 (local address 127.29.84.190:42275)
0806 00:47:53.113479 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.113484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.114640 (+ 1156us) server_negotiation.cc:407] Connection header received
0806 00:47:53.116600 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.116602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.116644 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.116713 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:53.117705 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.118551 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.120333 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.120551 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.120628 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.121857 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:53.121860 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:53.121862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:53.121873 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:53.121895 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:53.121920 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:53.121975 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.122187 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.122236 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:47:53.123489 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.113797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:42275 (local address 127.29.84.129:35537)
0806 00:47:53.114486 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:53.114542 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:53.116372 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:53.116898 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:53.116908 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:53.117010 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:53.117562 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.117574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.118688 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.118694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.120191 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.120207 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.120701 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.120707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.120920 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.121545 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:47:53.121549 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:53.121604 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:47:53.121686 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:53.122015 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:53.122028 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:53.122038 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 00:47:53.122175 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 00:47:53.122470 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":135,"threads_started":1}
I20260806 00:47:53.124714 27348 heartbeater.cc:347] Connected to a master server at 127.29.84.190:42275
I20260806 00:47:53.124939 27348 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:53.125526 27348 heartbeater.cc:511] Master 127.29.84.190:42275 requested a full tablet report, sending...
I20260806 00:47:53.127272 27040 ts_manager.cc:194] Registered new tserver with Master: 0d2fbd9b62c546ffa8362a6b52ba1eac (127.29.84.129:36637)
I20260806 00:47:53.128968 27040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.129:35537
W20260806 00:47:53.224076 27353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:53.224325 27353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:53.224395 27353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:53.228307 27353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:53.228376 27353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:53.228461 27353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.130
I20260806 00:47:53.232970 27353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.84.130
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.130
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:53.233860 27353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:53.235136 27353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:53.243889 27363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:53.243885 27364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:53.243948 27367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:53.244078 27353 server_base.cc:1034] running on GCE node
I20260806 00:47:53.244503 27353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:53.245096 27353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:53.246249 27353 hybrid_clock.cc:648] HybridClock initialized: now 1785977273246232 us; error 48 us; skew 500 ppm
I20260806 00:47:53.247959 27353 webserver.cc:460] Webserver started at http://127.29.84.130:34081/ using document root <none> and password file <none>
I20260806 00:47:53.248486 27353 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:53.248553 27353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:53.248708 27353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:53.250494 27353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/data/instance:
uuid: "f24dc72125ce49b9aab232aa76e013ba"
format_stamp: "Formatted at 2026-08-06 00:47:53 on dist-test-slave-n326"
server_key: "fb413bd2caccffcf94bf6259e6f5d653"
server_key_iv: "37eb3c537782b5db895d5cc478a053c3"
server_key_version: "encryptionkey@0"
I20260806 00:47:53.250988 27353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance:
uuid: "f24dc72125ce49b9aab232aa76e013ba"
format_stamp: "Formatted at 2026-08-06 00:47:53 on dist-test-slave-n326"
server_key: "fb413bd2caccffcf94bf6259e6f5d653"
server_key_iv: "37eb3c537782b5db895d5cc478a053c3"
server_key_version: "encryptionkey@0"
I20260806 00:47:53.254601 27353 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:47:53.256824 27379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:53.257938 27353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:47:53.258069 27353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/wal
uuid: "f24dc72125ce49b9aab232aa76e013ba"
format_stamp: "Formatted at 2026-08-06 00:47:53 on dist-test-slave-n326"
server_key: "fb413bd2caccffcf94bf6259e6f5d653"
server_key_iv: "37eb3c537782b5db895d5cc478a053c3"
server_key_version: "encryptionkey@0"
I20260806 00:47:53.258193 27353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:53.278360 27353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:53.278931 27353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:53.279160 27353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:53.279677 27353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:53.280568 27353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:53.280637 27353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:53.280704 27353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:53.280746 27353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:53.289335 27353 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.130:44943
I20260806 00:47:53.289383 27550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.130:44943 every 8 connection(s)
I20260806 00:47:53.290118 27353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/data/info.pb
I20260806 00:47:53.292721 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 27353
I20260806 00:47:53.292856 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/wal/instance
I20260806 00:47:53.293223 30034 external_mini_cluster.cc:1428] Setting key d16b11f8e0e6d5e5be954873ccdffc79
I20260806 00:47:53.295327 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.84.131:0
--local_ip_for_outbound_sockets=127.29.84.131
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42275
--builtin_ntp_servers=127.29.84.148:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:47:53.301247 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.291831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.130:50255 (local address 127.29.84.190:42275)
0806 00:47:53.291962 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.291965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.292995 (+ 1030us) server_negotiation.cc:407] Connection header received
0806 00:47:53.294995 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.294998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.295102 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.295206 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:53.296048 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.296884 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.298942 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.299351 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.299490 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.300596 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:53.300603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:53.300609 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:53.300629 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:53.300657 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:53.300670 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:53.300750 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.300962 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.301038 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:47:53.301962 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.292169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:42275 (local address 127.29.84.130:50255)
0806 00:47:53.292843 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:53.292896 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:53.294792 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:53.295343 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:53.295349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:53.295447 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:53.295906 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.295918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.297011 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.297015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.298788 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.298802 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.299515 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.299518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.299696 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.300322 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:47:53.300323 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:53.300362 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:47:53.300422 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:53.300780 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:53.300791 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:53.300799 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:47:53.300916 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 00:47:53.301130 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":114,"threads_started":1}
I20260806 00:47:53.303138 27553 heartbeater.cc:347] Connected to a master server at 127.29.84.190:42275
I20260806 00:47:53.303390 27553 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:53.303887 27553 heartbeater.cc:511] Master 127.29.84.190:42275 requested a full tablet report, sending...
I20260806 00:47:53.304960 27040 ts_manager.cc:194] Registered new tserver with Master: f24dc72125ce49b9aab232aa76e013ba (127.29.84.130:44943)
I20260806 00:47:53.305543 27040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.130:50255
W20260806 00:47:53.407629 27557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:53.407881 27557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:53.407944 27557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:53.412047 27557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:53.412115 27557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:53.412207 27557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.84.131
I20260806 00:47:53.416815 27557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.84.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.84.131
--webserver_port=0
--tserver_master_addrs=127.29.84.190:42275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.84.131
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:53.417785 27557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:53.419128 27557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:53.428303 27566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:53.428308 27565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:53.428313 27568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:53.428630 27557 server_base.cc:1034] running on GCE node
I20260806 00:47:53.429044 27557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:53.429627 27557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:53.430840 27557 hybrid_clock.cc:648] HybridClock initialized: now 1785977273430817 us; error 56 us; skew 500 ppm
I20260806 00:47:53.432699 27557 webserver.cc:460] Webserver started at http://127.29.84.131:33065/ using document root <none> and password file <none>
I20260806 00:47:53.433260 27557 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:53.433363 27557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:53.433607 27557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:53.435621 27557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/data/instance:
uuid: "0e50557837a544528a284a8d3f729074"
format_stamp: "Formatted at 2026-08-06 00:47:53 on dist-test-slave-n326"
server_key: "c6c08252c0d96067094ccdce7484b0b1"
server_key_iv: "a47af4fb4592401f811366c57534182f"
server_key_version: "encryptionkey@0"
I20260806 00:47:53.436152 27557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance:
uuid: "0e50557837a544528a284a8d3f729074"
format_stamp: "Formatted at 2026-08-06 00:47:53 on dist-test-slave-n326"
server_key: "c6c08252c0d96067094ccdce7484b0b1"
server_key_iv: "a47af4fb4592401f811366c57534182f"
server_key_version: "encryptionkey@0"
I20260806 00:47:53.439746 27557 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:47:53.442207 27579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:53.443535 27557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:53.443668 27557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/wal
uuid: "0e50557837a544528a284a8d3f729074"
format_stamp: "Formatted at 2026-08-06 00:47:53 on dist-test-slave-n326"
server_key: "c6c08252c0d96067094ccdce7484b0b1"
server_key_iv: "a47af4fb4592401f811366c57534182f"
server_key_version: "encryptionkey@0"
I20260806 00:47:53.443776 27557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:53.461335 27557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:53.461982 27557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:53.462168 27557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:53.462745 27557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:47:53.463611 27557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:47:53.463682 27557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:53.463747 27557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:47:53.463794 27557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:53.472432 27557 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.131:41259
I20260806 00:47:53.472462 27767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.131:41259 every 8 connection(s)
I20260806 00:47:53.473181 27557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/data/info.pb
I20260806 00:47:53.481506 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.474805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:36505 (local address 127.29.84.190:42275)
0806 00:47:53.474962 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.474966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.475973 (+ 1007us) server_negotiation.cc:407] Connection header received
0806 00:47:53.477238 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.477240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.477284 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.477354 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:53.478178 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.478697 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.479766 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.480007 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.480085 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.481089 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:53.481092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:53.481094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:53.481106 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:53.481122 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:53.481129 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:53.481196 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.481323 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.481365 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:47:53.482353 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.475212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:42275 (local address 127.29.84.131:36505)
0806 00:47:53.475823 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:53.475860 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:53.477065 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:53.477549 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:53.477555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:53.477634 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:47:53.478061 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.478072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.478808 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.478811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.479653 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.479662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.480134 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.480136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.480260 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.480866 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:47:53.480867 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:47:53.480903 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:47:53.480963 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:47:53.481214 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:47:53.481221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:47:53.481224 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:47:53.481306 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 00:47:53.481540 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":120,"threads_started":1}
I20260806 00:47:53.483377 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 27557
I20260806 00:47:53.483511 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/wal/instance
I20260806 00:47:53.483508 27769 heartbeater.cc:347] Connected to a master server at 127.29.84.190:42275
I20260806 00:47:53.483719 27769 heartbeater.cc:464] Registering TS with master...
I20260806 00:47:53.483858 30034 external_mini_cluster.cc:1428] Setting key eceaa878eaf34a4d2366e7e45eae9a9b
I20260806 00:47:53.484257 27769 heartbeater.cc:511] Master 127.29.84.190:42275 requested a full tablet report, sending...
I20260806 00:47:53.485419 27039 ts_manager.cc:194] Registered new tserver with Master: 0e50557837a544528a284a8d3f729074 (127.29.84.131:41259)
I20260806 00:47:53.486065 27039 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.84.131:36505
I20260806 00:47:53.497491 30034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:47:53.503225 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.499457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.29.84.190:42275)
0806 00:47:53.499594 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.499598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.499792 (+ 194us) server_negotiation.cc:407] Connection header received
0806 00:47:53.499938 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.499940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.499984 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.500070 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:53.500365 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.500853 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.501705 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.501914 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.502025 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.502794 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:53.502797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:53.502800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:53.502811 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:53.502835 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:53.502842 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:53.502918 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.503014 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.503084 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:47:53.505707 27039 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:47:53.508620 27039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:47:53.522281 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.516983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50004 (local address 127.29.84.129:36637)
0806 00:47:53.517445 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.517458 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.517482 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:47:53.517562 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.517566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.517732 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.517874 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:53.518237 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.519149 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.521056 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.521606 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.521729 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.521923 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.522021 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.522113 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":126,"threads_started":1}
I20260806 00:47:53.522284 27789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.516791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:36637 (local address 127.0.0.1:50004)
0806 00:47:53.517366 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:53.517381 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:53.517471 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:53.518011 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:53.518015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:53.518038 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:53.518119 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.518123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.519254 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.519258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.520945 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.520952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.521734 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.521738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.521842 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.521876 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 00:47:53.521995 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 00:47:53.522118 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":271,"thread_start_us":76,"threads_started":1}
I20260806 00:47:53.522342 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.516353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:44943 (local address 127.0.0.1:55616)
0806 00:47:53.517076 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:53.517090 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:53.517178 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:53.518365 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:53.518368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:53.518387 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:53.518480 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.518486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.519731 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.519737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.520833 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.520842 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.521861 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.521865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.521956 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.521977 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:47:53.522045 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 00:47:53.522155 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":47,"thread_start_us":106,"threads_started":1}
I20260806 00:47:53.522809 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.516228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.29.84.131:41259)
0806 00:47:53.516626 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.516639 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.516988 (+ 349us) server_negotiation.cc:407] Connection header received
0806 00:47:53.517274 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.517282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.517475 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.517604 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:53.518075 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.519228 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.520933 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.521710 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.521833 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.522000 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.522556 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.522662 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":108,"threads_started":1}
I20260806 00:47:53.523087 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.517281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55616 (local address 127.29.84.130:44943)
0806 00:47:53.517937 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.517948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.517970 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:47:53.518043 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.518047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.518219 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.518368 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:53.518609 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.519625 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.520985 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.521730 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.522516 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.522702 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.522749 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.522834 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":135,"threads_started":1}
I20260806 00:47:53.523181 27786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.516128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.131:41259 (local address 127.0.0.1:45166)
0806 00:47:53.516840 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:53.516901 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:53.517094 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:53.517790 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:53.517798 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:53.517826 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:53.517949 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.517956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.519341 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.519344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.520807 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.520816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.522064 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.522067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.522202 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.522224 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:47:53.522950 (+ 726us) client_negotiation.cc:240] Negotiation successful
0806 00:47:53.523023 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":189,"thread_start_us":70,"threads_started":1}
I20260806 00:47:53.525213 27238 tablet_service.cc:1467] Processing CreateTablet for tablet 028b2f2eec9b401a81ced9569fccfa6e (DEFAULT_TABLE table=test-table [id=83a06265bec44d4ab5ac5a449ad7e5c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:53.525373 27451 tablet_service.cc:1467] Processing CreateTablet for tablet 028b2f2eec9b401a81ced9569fccfa6e (DEFAULT_TABLE table=test-table [id=83a06265bec44d4ab5ac5a449ad7e5c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:53.526150 27238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 028b2f2eec9b401a81ced9569fccfa6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:53.526216 27451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 028b2f2eec9b401a81ced9569fccfa6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:53.531931 27797 tablet_bootstrap.cc:492] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac: Bootstrap starting.
I20260806 00:47:53.532699 27798 tablet_bootstrap.cc:492] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: Bootstrap starting.
I20260806 00:47:53.533336 27651 tablet_service.cc:1467] Processing CreateTablet for tablet 028b2f2eec9b401a81ced9569fccfa6e (DEFAULT_TABLE table=test-table [id=83a06265bec44d4ab5ac5a449ad7e5c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:47:53.534097 27651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 028b2f2eec9b401a81ced9569fccfa6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:53.534169 27797 tablet_bootstrap.cc:654] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:53.534723 27798 tablet_bootstrap.cc:654] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:53.535024 27797 log.cc:824] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:53.535550 27798 log.cc:824] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:53.537988 27798 tablet_bootstrap.cc:492] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: No bootstrap required, opened a new log
I20260806 00:47:53.537988 27797 tablet_bootstrap.cc:492] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac: No bootstrap required, opened a new log
I20260806 00:47:53.538182 27798 ts_tablet_manager.cc:1397] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 00:47:53.538182 27797 ts_tablet_manager.cc:1397] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:47:53.539922 27801 tablet_bootstrap.cc:492] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: Bootstrap starting.
I20260806 00:47:53.541980 27801 tablet_bootstrap.cc:654] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:53.542055 27798 raft_consensus.cc:348] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } }
I20260806 00:47:53.542055 27797 raft_consensus.cc:348] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } }
I20260806 00:47:53.542311 27797 raft_consensus.cc:374] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:53.542311 27798 raft_consensus.cc:374] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:53.542379 27797 raft_consensus.cc:729] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d2fbd9b62c546ffa8362a6b52ba1eac, State: Initialized, Role: FOLLOWER
I20260806 00:47:53.542371 27798 raft_consensus.cc:729] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24dc72125ce49b9aab232aa76e013ba, State: Initialized, Role: FOLLOWER
I20260806 00:47:53.542773 27801 log.cc:824] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:53.542825 27798 consensus_queue.cc:260] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } }
I20260806 00:47:53.542825 27797 consensus_queue.cc:260] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } }
I20260806 00:47:53.543464 27553 heartbeater.cc:503] Master 127.29.84.190:42275 was elected leader, sending a full tablet report...
I20260806 00:47:53.543802 27797 ts_tablet_manager.cc:1428] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260806 00:47:53.543802 27798 ts_tablet_manager.cc:1428] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260806 00:47:53.544142 27348 heartbeater.cc:503] Master 127.29.84.190:42275 was elected leader, sending a full tablet report...
I20260806 00:47:53.545409 27801 tablet_bootstrap.cc:492] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: No bootstrap required, opened a new log
I20260806 00:47:53.545609 27801 ts_tablet_manager.cc:1397] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260806 00:47:53.549392 27801 raft_consensus.cc:348] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } }
I20260806 00:47:53.549571 27801 raft_consensus.cc:374] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:53.549643 27801 raft_consensus.cc:729] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e50557837a544528a284a8d3f729074, State: Initialized, Role: FOLLOWER
I20260806 00:47:53.549979 27801 consensus_queue.cc:260] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } }
I20260806 00:47:53.550537 27769 heartbeater.cc:503] Master 127.29.84.190:42275 was elected leader, sending a full tablet report...
I20260806 00:47:53.550817 27801 ts_tablet_manager.cc:1428] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
W20260806 00:47:53.612650 27350 tablet.cc:2369] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:47:53.670518 27809 raft_consensus.cc:482] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:47:53.670684 27809 raft_consensus.cc:504] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } }
I20260806 00:47:53.671692 27809 leader_election.cc:288] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f24dc72125ce49b9aab232aa76e013ba (127.29.84.130:44943), 0d2fbd9b62c546ffa8362a6b52ba1eac (127.29.84.129:36637)
I20260806 00:47:53.676007 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.671958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.130:44943 (local address 127.29.84.131:52931)
0806 00:47:53.672101 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:53.672118 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:53.672221 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:53.672572 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:53.672575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:53.672597 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:53.672728 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.672734 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.673696 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.673698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.674873 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.674884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.675587 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.675591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.675701 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.675730 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 00:47:53.675803 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:47:53.675884 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 00:47:53.676048 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.672027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:52931 (local address 127.29.84.130:44943)
0806 00:47:53.672151 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.672155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.672168 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:47:53.672334 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.672340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.672397 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.672496 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:53.672830 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.673587 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.675008 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.675469 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.675565 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.675604 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.675839 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.675912 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:47:53.676187 27810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.672283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:36637 (local address 127.29.84.131:55061)
0806 00:47:53.672631 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:53.672644 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:47:53.672734 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:47:53.672974 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:47:53.672975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:47:53.672991 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:47:53.673084 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.673088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.673927 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.673930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.675047 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:47:53.675054 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.675751 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.675754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:47:53.675873 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.675905 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 00:47:53.675981 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 00:47:53.676044 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":122,"threads_started":1}
I20260806 00:47:53.676240 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.672401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.84.131:55061 (local address 127.29.84.129:36637)
0806 00:47:53.672538 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.672542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.672719 (+ 177us) server_negotiation.cc:407] Connection header received
0806 00:47:53.672829 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.672832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.672880 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.672960 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:47:53.673193 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.673836 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.675204 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.675631 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.675746 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.675805 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.676024 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.676097 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:47:53.676739 27267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "028b2f2eec9b401a81ced9569fccfa6e" candidate_uuid: "0e50557837a544528a284a8d3f729074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" is_pre_election: true
I20260806 00:47:53.676744 27483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "028b2f2eec9b401a81ced9569fccfa6e" candidate_uuid: "0e50557837a544528a284a8d3f729074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24dc72125ce49b9aab232aa76e013ba" is_pre_election: true
I20260806 00:47:53.676992 27267 raft_consensus.cc:2454] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e50557837a544528a284a8d3f729074 in term 0.
I20260806 00:47:53.677008 27483 raft_consensus.cc:2454] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e50557837a544528a284a8d3f729074 in term 0.
I20260806 00:47:53.677476 27580 leader_election.cc:302] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e50557837a544528a284a8d3f729074, f24dc72125ce49b9aab232aa76e013ba; no voters:
I20260806 00:47:53.677745 27809 raft_consensus.cc:2781] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:47:53.677845 27809 raft_consensus.cc:482] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:47:53.677912 27809 raft_consensus.cc:3037] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:53.678933 27809 raft_consensus.cc:504] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } }
I20260806 00:47:53.679271 27809 leader_election.cc:288] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [CANDIDATE]: Term 1 election: Requested vote from peers f24dc72125ce49b9aab232aa76e013ba (127.29.84.130:44943), 0d2fbd9b62c546ffa8362a6b52ba1eac (127.29.84.129:36637)
I20260806 00:47:53.679605 27483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "028b2f2eec9b401a81ced9569fccfa6e" candidate_uuid: "0e50557837a544528a284a8d3f729074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24dc72125ce49b9aab232aa76e013ba"
I20260806 00:47:53.679697 27267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "028b2f2eec9b401a81ced9569fccfa6e" candidate_uuid: "0e50557837a544528a284a8d3f729074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac"
I20260806 00:47:53.679742 27483 raft_consensus.cc:3037] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:53.679850 27267 raft_consensus.cc:3037] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:53.680914 27483 raft_consensus.cc:2454] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e50557837a544528a284a8d3f729074 in term 1.
I20260806 00:47:53.680927 27267 raft_consensus.cc:2454] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e50557837a544528a284a8d3f729074 in term 1.
I20260806 00:47:53.681257 27580 leader_election.cc:302] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e50557837a544528a284a8d3f729074, f24dc72125ce49b9aab232aa76e013ba; no voters:
I20260806 00:47:53.681437 27809 raft_consensus.cc:2781] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:53.681680 27809 raft_consensus.cc:686] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 1 LEADER]: Becoming Leader. State: Replica: 0e50557837a544528a284a8d3f729074, State: Running, Role: LEADER
I20260806 00:47:53.682009 27809 consensus_queue.cc:237] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } }
I20260806 00:47:53.685019 27040 catalog_manager.cc:5495] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e50557837a544528a284a8d3f729074 (127.29.84.131). New cstate: current_term: 1 leader_uuid: "0e50557837a544528a284a8d3f729074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e50557837a544528a284a8d3f729074" member_type: VOTER last_known_addr { host: "127.29.84.131" port: 41259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 } health_report { overall_health: UNKNOWN } } }
I20260806 00:47:53.709930 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.706859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.29.84.131:41259)
0806 00:47:53.706979 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:53.706982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:53.706994 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:47:53.707162 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:53.707164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:53.707213 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:53.707296 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:47:53.707529 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.707989 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.708775 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:53.708963 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:53.709032 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:53.709265 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:47:53.709615 (+ 350us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:47:53.709702 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:53.709752 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 00:47:53.709790 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:47:53.715204 27267 raft_consensus.cc:1261] T 028b2f2eec9b401a81ced9569fccfa6e P 0d2fbd9b62c546ffa8362a6b52ba1eac [term 1 FOLLOWER]: Refusing update from remote peer 0e50557837a544528a284a8d3f729074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:53.715224 27483 raft_consensus.cc:1261] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [term 1 FOLLOWER]: Refusing update from remote peer 0e50557837a544528a284a8d3f729074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:47:53.715857 27809 consensus_queue.cc:1035] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d2fbd9b62c546ffa8362a6b52ba1eac" member_type: VOTER last_known_addr { host: "127.29.84.129" port: 36637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:53.716035 27811 consensus_queue.cc:1035] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24dc72125ce49b9aab232aa76e013ba" member_type: VOTER last_known_addr { host: "127.29.84.130" port: 44943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:47:53.723032 27817 mvcc.cc:205] Tried to move back new op lower bound from 7315362913131053056 to 7315362913004240896. Current Snapshot: MvccSnapshot[applied={T|T < 7315362913131053056}]
I20260806 00:47:53.725914 27816 mvcc.cc:205] Tried to move back new op lower bound from 7315362913131053056 to 7315362913004240896. Current Snapshot: MvccSnapshot[applied={T|T < 7315362913131053056}]
I20260806 00:47:53.731545 27040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:53.731669 27040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:47:53.733794 27040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0745783ead24ce192b5482a2ff1a62f: Sending DeleteTablet for 3 replicas of tablet 028b2f2eec9b401a81ced9569fccfa6e
I20260806 00:47:53.734493 27651 tablet_service.cc:1514] Processing DeleteTablet for tablet 028b2f2eec9b401a81ced9569fccfa6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:53 UTC) from {username='slave'} at 127.0.0.1:45166
I20260806 00:47:53.734795 27238 tablet_service.cc:1514] Processing DeleteTablet for tablet 028b2f2eec9b401a81ced9569fccfa6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:53 UTC) from {username='slave'} at 127.0.0.1:50004
I20260806 00:47:53.734814 27451 tablet_service.cc:1514] Processing DeleteTablet for tablet 028b2f2eec9b401a81ced9569fccfa6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:47:53 UTC) from {username='slave'} at 127.0.0.1:55616
I20260806 00:47:53.734942 27827 tablet_replica.cc:330] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: stopping tablet replica
I20260806 00:47:53.735234 27827 raft_consensus.cc:2229] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:47:53.735523 27833 tablet_replica.cc:330] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: stopping tablet replica
I20260806 00:47:53.735556 27827 raft_consensus.cc:2258] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:53.735610 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 27133
I20260806 00:47:53.740863 27827 ts_tablet_manager.cc:1905] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:53.744390 27827 ts_tablet_manager.cc:1918] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:53.744472 27827 log.cc:1196] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-2/wal/wals/028b2f2eec9b401a81ced9569fccfa6e
I20260806 00:47:53.744760 27827 ts_tablet_manager.cc:1939] T 028b2f2eec9b401a81ced9569fccfa6e P 0e50557837a544528a284a8d3f729074: Deleting consensus metadata
I20260806 00:47:53.746073 27006 catalog_manager.cc:4841] TS 0e50557837a544528a284a8d3f729074 (127.29.84.131:41259): tablet 028b2f2eec9b401a81ced9569fccfa6e (table test-table [id=83a06265bec44d4ab5ac5a449ad7e5c4]) successfully deleted
I20260806 00:47:53.747383 27833 raft_consensus.cc:2229] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:47:53.747740 27833 raft_consensus.cc:2258] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:47:53.748983 27833 ts_tablet_manager.cc:1905] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:47:53.751137 27833 ts_tablet_manager.cc:1918] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:47:53.751255 27833 log.cc:1196] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: Deleting WAL directory at /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977020829091-30034-0/minicluster-data/ts-1/wal/wals/028b2f2eec9b401a81ced9569fccfa6e
I20260806 00:47:53.751577 27833 ts_tablet_manager.cc:1939] T 028b2f2eec9b401a81ced9569fccfa6e P f24dc72125ce49b9aab232aa76e013ba: Deleting consensus metadata
I20260806 00:47:53.752544 27005 catalog_manager.cc:4841] TS f24dc72125ce49b9aab232aa76e013ba (127.29.84.130:44943): tablet 028b2f2eec9b401a81ced9569fccfa6e (table test-table [id=83a06265bec44d4ab5ac5a449ad7e5c4]) successfully deleted
W20260806 00:47:53.754290 27005 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.84.129:36637 (error 108)
I20260806 00:47:53.755141 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 27353
I20260806 00:47:53.755295 27786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:53.754962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.129:36637 (local address 127.0.0.1:50012)
0806 00:47:53.755145 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 00:47:53.755217 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.129:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 00:47:53.764356 27005 catalog_manager.cc:4584] TS 0d2fbd9b62c546ffa8362a6b52ba1eac (127.29.84.129:36637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 028b2f2eec9b401a81ced9569fccfa6e: Network error: Client connection negotiation failed: client connection to 127.29.84.129:36637: connect: Connection refused (error 111)
I20260806 00:47:53.770844 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 27557
I20260806 00:47:53.778277 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 26979
2026-08-06T00:47:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1145 ms)
[----------] 6 tests from EncryptionPolicyTest (10695 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:47:53 dist-test-slave-n326 krb5kdc[27839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:47:53 dist-test-slave-n326 krb5kdc[27839](info): set up 2 sockets
Aug 06 00:47:53 dist-test-slave-n326 krb5kdc[27839](info): commencing operation
krb5kdc: starting...
W20260806 00:47:55.008407 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:47:55 dist-test-slave-n326 krb5kdc[27839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:47:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:47:55.127496 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:55.232108 27859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:55.232352 27859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:55.232425 27859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:55.236202 27859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:55.236269 27859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:55.236305 27859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:55.236354 27859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:55.236389 27859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:55.241261 27859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45009
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:55.242170 27859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:55.243116 27859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:55.250998 27874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:55.251015 27868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:55.250998 27872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:55.251291 27859 server_base.cc:1034] running on GCE node
I20260806 00:47:55.251914 27859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:55.252910 27859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:55.254096 27859 hybrid_clock.cc:648] HybridClock initialized: now 1785977275254082 us; error 33 us; skew 500 ppm
Aug 06 00:47:55 dist-test-slave-n326 krb5kdc[27839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:55.256647 27859 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:47:55.257444 27859 webserver.cc:460] Webserver started at http://127.0.0.1:34197/ using document root <none> and password file <none>
I20260806 00:47:55.257958 27859 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:55.258052 27859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:55.258271 27859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:55.260207 27859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "76741f0f01024c9eab313138af6c57e4"
format_stamp: "Formatted at 2026-08-06 00:47:55 on dist-test-slave-n326"
server_key: "18594d0c45ba4e4c8c3592f0afe26952"
server_key_iv: "9e7f4fa542641d337d670df1dad37db0"
server_key_version: "encryptionkey@0"
I20260806 00:47:55.260746 27859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "76741f0f01024c9eab313138af6c57e4"
format_stamp: "Formatted at 2026-08-06 00:47:55 on dist-test-slave-n326"
server_key: "18594d0c45ba4e4c8c3592f0afe26952"
server_key_iv: "9e7f4fa542641d337d670df1dad37db0"
server_key_version: "encryptionkey@0"
I20260806 00:47:55.264112 27859 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:47:55.266201 27885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:55.267303 27859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:47:55.267427 27859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "76741f0f01024c9eab313138af6c57e4"
format_stamp: "Formatted at 2026-08-06 00:47:55 on dist-test-slave-n326"
server_key: "18594d0c45ba4e4c8c3592f0afe26952"
server_key_iv: "9e7f4fa542641d337d670df1dad37db0"
server_key_version: "encryptionkey@0"
I20260806 00:47:55.267557 27859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:55.308040 27859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:55.310554 27859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:55.310756 27859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:55.317698 27859 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45009
I20260806 00:47:55.317713 27962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45009 every 8 connection(s)
I20260806 00:47:55.318562 27859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:55.321122 27963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:55.323307 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 27859
I20260806 00:47:55.323455 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:55.323767 30034 external_mini_cluster.cc:1428] Setting key 327367266f906466a61fb8da85c84378
I20260806 00:47:55.326889 27963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4: Bootstrap starting.
I20260806 00:47:55.329296 27963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:55.330044 27963 log.cc:824] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:55.332201 27963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4: No bootstrap required, opened a new log
Aug 06 00:47:55 dist-test-slave-n326 krb5kdc[27839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:47:55.335507 27963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76741f0f01024c9eab313138af6c57e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } }
I20260806 00:47:55.335773 27963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:55.335862 27963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76741f0f01024c9eab313138af6c57e4, State: Initialized, Role: FOLLOWER
I20260806 00:47:55.336325 27963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76741f0f01024c9eab313138af6c57e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } }
I20260806 00:47:55.336488 27963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:55.336577 27963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:55.336674 27963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:55.337584 27963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76741f0f01024c9eab313138af6c57e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } }
I20260806 00:47:55.337901 27963 leader_election.cc:302] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76741f0f01024c9eab313138af6c57e4; no voters:
I20260806 00:47:55.338225 27963 leader_election.cc:288] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:47:55.338236 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:55.325034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36675 (local address 127.0.0.1:45009)
0806 00:47:55.325387 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:55.325401 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:55.325432 (+ 31us) server_negotiation.cc:407] Connection header received
0806 00:47:55.325983 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:55.325997 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:55.326276 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:55.326566 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:55.327477 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:55.328539 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:55.329674 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:55.330121 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:55.330277 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:55.333014 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:55.333018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:55.333030 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:55.333053 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:55.335164 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:55.335722 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:55.335727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:55.335731 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:55.335899 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:55.336219 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:55.336221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:55.336223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:55.336681 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:55.336909 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:55.337246 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 00:47:55.337447 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":108,"threads_started":1}
I20260806 00:47:55.338294 27969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:55.338515 27969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [term 1 LEADER]: Becoming Leader. State: Replica: 76741f0f01024c9eab313138af6c57e4, State: Running, Role: LEADER
I20260806 00:47:55.338790 27969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76741f0f01024c9eab313138af6c57e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } }
I20260806 00:47:55.339466 27963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:55.339972 27969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76741f0f01024c9eab313138af6c57e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76741f0f01024c9eab313138af6c57e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } } }
I20260806 00:47:55.340086 27969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:55.340291 27969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76741f0f01024c9eab313138af6c57e4. Latest consensus state: current_term: 1 leader_uuid: "76741f0f01024c9eab313138af6c57e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76741f0f01024c9eab313138af6c57e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } } }
I20260806 00:47:55.340389 27969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4 [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:55.341908 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 00:47:55.344576 27991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:47:55.344658 27991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:47:55.344739 27977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:55.345434 27977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:55.351171 27977 catalog_manager.cc:1347] Generated new cluster ID: 5793f5f51bcc44e6ab59f960b4db5c9e
I20260806 00:47:55.351239 27977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:55.352504 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:55.344044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57269 (local address 127.0.0.1:45009)
0806 00:47:55.344134 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:55.344138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:55.344525 (+ 387us) server_negotiation.cc:407] Connection header received
0806 00:47:55.344711 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:55.344714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:55.344758 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:55.344808 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:55.345581 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:55.346066 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:55.346879 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:55.347243 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:55.347344 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:55.348987 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:55.348991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:55.348994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:47:55.349017 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:55.350499 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:55.350980 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:55.350984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:55.350985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:55.351125 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:47:55.351559 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:47:55.351562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:47:55.351564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:47:55.351817 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:55.351991 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:55.352257 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 00:47:55.352355 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 00:47:55.379184 27977 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:47:55.379973 27977 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:55.386664 27977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76741f0f01024c9eab313138af6c57e4: Generated new TSK 0
I20260806 00:47:55.387343 27977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:47:55.398708 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 27859
W20260806 00:47:55.413414 27836 connection.cc:535] client connection to 127.0.0.1:45009 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45009): Connection reset by peer (error 104)
2026-08-06T00:47:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1624 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:47:55 dist-test-slave-n326 krb5kdc[28006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:47:55 dist-test-slave-n326 krb5kdc[28006](info): set up 2 sockets
Aug 06 00:47:55 dist-test-slave-n326 krb5kdc[28006](info): commencing operation
krb5kdc: starting...
W20260806 00:47:56.605844 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:47:56 dist-test-slave-n326 krb5kdc[28006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:47:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:47:56.725956 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:56.830861 28032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:56.831141 28032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:56.831247 28032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:56.834961 28032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:56.835057 28032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:56.835086 28032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:56.835140 28032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:56.835181 28032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:56.840082 28032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:56.841001 28032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:56.841943 28032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:56.849810 28046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:56.849810 28037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:56.849972 28041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:56.850071 28032 server_base.cc:1034] running on GCE node
I20260806 00:47:56.850746 28032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:56.851814 28032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:56.852993 28032 hybrid_clock.cc:648] HybridClock initialized: now 1785977276852975 us; error 29 us; skew 500 ppm
Aug 06 00:47:56 dist-test-slave-n326 krb5kdc[28006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977276, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:56.855592 28032 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:47:56.856318 28032 webserver.cc:460] Webserver started at http://127.0.0.1:38255/ using document root <none> and password file <none>
I20260806 00:47:56.856835 28032 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:56.856921 28032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:56.857134 28032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:56.858999 28032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "57667dcb652a43ccaa0b788b2887ed82"
format_stamp: "Formatted at 2026-08-06 00:47:56 on dist-test-slave-n326"
server_key: "2dc7219161a762801bcc52d3d2227379"
server_key_iv: "edffa865852843b40c5112e4be8d1923"
server_key_version: "encryptionkey@0"
I20260806 00:47:56.859611 28032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "57667dcb652a43ccaa0b788b2887ed82"
format_stamp: "Formatted at 2026-08-06 00:47:56 on dist-test-slave-n326"
server_key: "2dc7219161a762801bcc52d3d2227379"
server_key_iv: "edffa865852843b40c5112e4be8d1923"
server_key_version: "encryptionkey@0"
I20260806 00:47:56.862987 28032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:47:56.865227 28059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:56.866396 28032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:47:56.866564 28032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "57667dcb652a43ccaa0b788b2887ed82"
format_stamp: "Formatted at 2026-08-06 00:47:56 on dist-test-slave-n326"
server_key: "2dc7219161a762801bcc52d3d2227379"
server_key_iv: "edffa865852843b40c5112e4be8d1923"
server_key_version: "encryptionkey@0"
I20260806 00:47:56.866673 28032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:56.876631 28032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:56.879729 28032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:56.879913 28032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:56.886783 28032 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39491
I20260806 00:47:56.886819 28147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39491 every 8 connection(s)
I20260806 00:47:56.887739 28032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:56.890312 28151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:56.891485 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 28032
I20260806 00:47:56.891639 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:56.891958 30034 external_mini_cluster.cc:1428] Setting key 07ed0bbb4b8d48aa31e678f9f8085953
I20260806 00:47:56.896193 28151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82: Bootstrap starting.
I20260806 00:47:56.898301 28151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:56.899191 28151 log.cc:824] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:56.900801 28155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:56.893305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45857 (local address 127.0.0.1:39491)
0806 00:47:56.893741 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:56.893756 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:56.893788 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:47:56.894384 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:56.894399 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:56.894711 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:56.895021 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:56.895430 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:56.896419 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:56.897320 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:56.897778 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:56.897960 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:56.898815 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:56.898819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:56.898831 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:56.899300 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:56.899370 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:56.899395 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:56.899494 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:56.899744 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 00:47:56.899960 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":149,"threads_started":1}
I20260806 00:47:56.901283 28151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82: No bootstrap required, opened a new log
I20260806 00:47:56.904469 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:47:56.905699 28151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57667dcb652a43ccaa0b788b2887ed82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260806 00:47:56.905987 28151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:56.906064 28151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57667dcb652a43ccaa0b788b2887ed82, State: Initialized, Role: FOLLOWER
I20260806 00:47:56.906509 28151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57667dcb652a43ccaa0b788b2887ed82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260806 00:47:56.906683 28151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:56.906759 28151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:56.906857 28151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:56.908007 28151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57667dcb652a43ccaa0b788b2887ed82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260806 00:47:56.908344 28151 leader_election.cc:302] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57667dcb652a43ccaa0b788b2887ed82; no voters:
I20260806 00:47:56.908699 28151 leader_election.cc:288] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:47:56.908764 28165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:56.908995 28165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [term 1 LEADER]: Becoming Leader. State: Replica: 57667dcb652a43ccaa0b788b2887ed82, State: Running, Role: LEADER
I20260806 00:47:56.909271 28165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57667dcb652a43ccaa0b788b2887ed82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260806 00:47:56.909981 28151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:56.911175 28166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57667dcb652a43ccaa0b788b2887ed82. Latest consensus state: current_term: 1 leader_uuid: "57667dcb652a43ccaa0b788b2887ed82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57667dcb652a43ccaa0b788b2887ed82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } } }
I20260806 00:47:56.911209 28167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57667dcb652a43ccaa0b788b2887ed82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57667dcb652a43ccaa0b788b2887ed82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } } }
I20260806 00:47:56.911304 28166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:56.911314 28167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82 [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:56.912318 28155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:56.906695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49903 (local address 127.0.0.1:39491)
0806 00:47:56.906815 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:56.906819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:56.907087 (+ 268us) server_negotiation.cc:407] Connection header received
0806 00:47:56.907210 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:56.907213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:56.907254 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:56.907333 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:56.907566 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:56.908316 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:56.909324 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:47:56.909617 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:47:56.909675 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:47:56.911653 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:56.911656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:56.911659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:56.911671 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:56.911694 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:56.911706 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:56.911794 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:56.912165 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 00:47:56.912201 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 00:47:56.915015 28186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:47:56.915126 28186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:47:56.915217 28177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:56.915880 28177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:56.920693 28177 catalog_manager.cc:1347] Generated new cluster ID: c02aee2a0b074f4c996d415b60f644b1
I20260806 00:47:56.920742 28177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:56.932472 28177 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:47:56.933579 28177 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:56.940707 28177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57667dcb652a43ccaa0b788b2887ed82: Generated new TSK 0
I20260806 00:47:56.941282 28177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:47:56.944496 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 28032
W20260806 00:47:56.960086 28004 connection.cc:535] client connection to 127.0.0.1:39491 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39491): Connection reset by peer (error 104)
2026-08-06T00:47:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:47:57 dist-test-slave-n326 krb5kdc[28196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:47:57 dist-test-slave-n326 krb5kdc[28196](info): set up 2 sockets
Aug 06 00:47:57 dist-test-slave-n326 krb5kdc[28196](info): commencing operation
krb5kdc: starting...
W20260806 00:47:58.163108 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:47:58 dist-test-slave-n326 krb5kdc[28196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:47:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:47:58.283551 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:58.390385 28220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:58.390614 28220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:58.390653 28220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:58.394608 28220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:58.394657 28220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:58.394675 28220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:58.394691 28220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:58.394706 28220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:58.399684 28220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:58.400607 28220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:58.401594 28220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:58.409538 28234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:58.409530 28228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:58.409600 28226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:58.409927 28220 server_base.cc:1034] running on GCE node
I20260806 00:47:58.410368 28220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:58.411392 28220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:58.412559 28220 hybrid_clock.cc:648] HybridClock initialized: now 1785977278412532 us; error 39 us; skew 500 ppm
Aug 06 00:47:58 dist-test-slave-n326 krb5kdc[28196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977278, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:58.419689 28220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:47:58.420423 28220 webserver.cc:460] Webserver started at http://127.0.0.1:41257/ using document root <none> and password file <none>
I20260806 00:47:58.420885 28220 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:58.420939 28220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:58.421084 28220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:58.422919 28220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "ec6a7056c6214ad29e27d13e97b24e1e"
format_stamp: "Formatted at 2026-08-06 00:47:58 on dist-test-slave-n326"
server_key: "a0eb363c34800371b34c3cc19dd5227d"
server_key_iv: "17598706faee7468852e98097a2f59bf"
server_key_version: "encryptionkey@0"
I20260806 00:47:58.423454 28220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "ec6a7056c6214ad29e27d13e97b24e1e"
format_stamp: "Formatted at 2026-08-06 00:47:58 on dist-test-slave-n326"
server_key: "a0eb363c34800371b34c3cc19dd5227d"
server_key_iv: "17598706faee7468852e98097a2f59bf"
server_key_version: "encryptionkey@0"
I20260806 00:47:58.426684 28220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:47:58.428870 28245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:58.429945 28220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:47:58.430064 28220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "ec6a7056c6214ad29e27d13e97b24e1e"
format_stamp: "Formatted at 2026-08-06 00:47:58 on dist-test-slave-n326"
server_key: "a0eb363c34800371b34c3cc19dd5227d"
server_key_iv: "17598706faee7468852e98097a2f59bf"
server_key_version: "encryptionkey@0"
I20260806 00:47:58.430155 28220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:58.434293 28220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:58.437213 28220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:58.437345 28220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:58.443517 28220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41635
I20260806 00:47:58.443593 28321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41635 every 8 connection(s)
I20260806 00:47:58.444357 28220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:58.446843 28322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:58.448978 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 28220
I20260806 00:47:58.449131 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:58.449432 30034 external_mini_cluster.cc:1428] Setting key 8ac11c161eaa295b996616ebb7ff0857
I20260806 00:47:58.452780 28322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e: Bootstrap starting.
I20260806 00:47:58.454638 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:58.450743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33925 (local address 127.0.0.1:41635)
0806 00:47:58.451158 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:58.451177 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:58.451217 (+ 40us) server_negotiation.cc:407] Connection header received
0806 00:47:58.452008 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:58.452029 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:58.452363 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:58.452708 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:58.453353 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:58.453358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:58.453371 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:58.453382 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:58.453443 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:58.453465 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:58.453591 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:58.453651 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 00:47:58.453800 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":104,"threads_started":1}
I20260806 00:47:58.455360 28322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e: Neither blocks nor log segments found. Creating new log.
I20260806 00:47:58.456194 28322 log.cc:824] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:47:58.457463 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:47:58.458137 28322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e: No bootstrap required, opened a new log
I20260806 00:47:58.460978 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:58.459539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57727 (local address 127.0.0.1:41635)
0806 00:47:58.459647 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:58.459650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:58.459800 (+ 150us) server_negotiation.cc:407] Connection header received
0806 00:47:58.459935 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:58.459942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:58.459967 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:58.460044 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:47:58.460644 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:47:58.460647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:47:58.460649 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:47:58.460659 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:47:58.460682 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:47:58.460698 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:47:58.460779 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:47:58.460846 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 00:47:58.460879 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:47:58.461731 28322 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6a7056c6214ad29e27d13e97b24e1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } }
I20260806 00:47:58.461998 28322 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:47:58.462060 28322 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec6a7056c6214ad29e27d13e97b24e1e, State: Initialized, Role: FOLLOWER
I20260806 00:47:58.462484 28322 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6a7056c6214ad29e27d13e97b24e1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } }
I20260806 00:47:58.462638 28322 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:47:58.462702 28322 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:47:58.462795 28322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:47:58.463596 28322 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6a7056c6214ad29e27d13e97b24e1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } }
I20260806 00:47:58.463892 28322 leader_election.cc:302] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec6a7056c6214ad29e27d13e97b24e1e; no voters:
I20260806 00:47:58.464181 28322 leader_election.cc:288] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:47:58.464237 28341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:47:58.464416 28341 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [term 1 LEADER]: Becoming Leader. State: Replica: ec6a7056c6214ad29e27d13e97b24e1e, State: Running, Role: LEADER
I20260806 00:47:58.464660 28341 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6a7056c6214ad29e27d13e97b24e1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } }
I20260806 00:47:58.465332 28322 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:47:58.466303 28342 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec6a7056c6214ad29e27d13e97b24e1e. Latest consensus state: current_term: 1 leader_uuid: "ec6a7056c6214ad29e27d13e97b24e1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6a7056c6214ad29e27d13e97b24e1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } } }
I20260806 00:47:58.466352 28343 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec6a7056c6214ad29e27d13e97b24e1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6a7056c6214ad29e27d13e97b24e1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } } }
I20260806 00:47:58.466432 28342 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:58.466455 28343 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e [sys.catalog]: This master's current role is: LEADER
I20260806 00:47:58.466835 28358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:47:58.469720 28358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:47:58.474706 28358 catalog_manager.cc:1347] Generated new cluster ID: 675b83ad011b48f495efdcb51d994dd3
I20260806 00:47:58.474776 28358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:47:58.496026 28358 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:47:58.496321 28358 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:47:58.502345 28358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec6a7056c6214ad29e27d13e97b24e1e: Generated new TSK 0
I20260806 00:47:58.502884 28358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:47:58.509963 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 28220
2026-08-06T00:47:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:47:58 dist-test-slave-n326 krb5kdc[28373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:47:58 dist-test-slave-n326 krb5kdc[28373](info): set up 2 sockets
Aug 06 00:47:58 dist-test-slave-n326 krb5kdc[28373](info): commencing operation
krb5kdc: starting...
W20260806 00:47:59.706213 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:47:59 dist-test-slave-n326 krb5kdc[28373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:47:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:47:59.826349 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:47:59.932003 28392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:47:59.932230 28392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:47:59.932268 28392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:47:59.936079 28392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:47:59.936126 28392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:47:59.936146 28392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:47:59.936160 28392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:47:59.936175 28392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:47:59.941192 28392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:47:59.942150 28392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:47:59.942989 28392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:47:59.950799 28407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:59.950793 28401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:47:59.950819 28402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:47:59.950918 28392 server_base.cc:1034] running on GCE node
I20260806 00:47:59.951601 28392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:47:59.952590 28392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:47:59.953799 28392 hybrid_clock.cc:648] HybridClock initialized: now 1785977279953772 us; error 37 us; skew 500 ppm
Aug 06 00:47:59 dist-test-slave-n326 krb5kdc[28373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:47:59.956449 28392 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:47:59.957221 28392 webserver.cc:460] Webserver started at http://127.0.0.1:34313/ using document root <none> and password file <none>
I20260806 00:47:59.957715 28392 fs_manager.cc:359] Metadata directory not provided
I20260806 00:47:59.957803 28392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:47:59.958046 28392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:47:59.959924 28392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "9f03ea1dfe344d1996bb8665023de208"
format_stamp: "Formatted at 2026-08-06 00:47:59 on dist-test-slave-n326"
server_key: "fdef83913b00ce5ddda8f0514d54d845"
server_key_iv: "cc64fe2875542aaa1908047e590e1d70"
server_key_version: "encryptionkey@0"
I20260806 00:47:59.960454 28392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "9f03ea1dfe344d1996bb8665023de208"
format_stamp: "Formatted at 2026-08-06 00:47:59 on dist-test-slave-n326"
server_key: "fdef83913b00ce5ddda8f0514d54d845"
server_key_iv: "cc64fe2875542aaa1908047e590e1d70"
server_key_version: "encryptionkey@0"
I20260806 00:47:59.963846 28392 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:47:59.966130 28422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:47:59.967481 28392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:47:59.967636 28392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "9f03ea1dfe344d1996bb8665023de208"
format_stamp: "Formatted at 2026-08-06 00:47:59 on dist-test-slave-n326"
server_key: "fdef83913b00ce5ddda8f0514d54d845"
server_key_iv: "cc64fe2875542aaa1908047e590e1d70"
server_key_version: "encryptionkey@0"
I20260806 00:47:59.967744 28392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:47:59.982739 28392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:47:59.985287 28392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:47:59.985466 28392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:47:59.991742 28392 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33115
I20260806 00:47:59.991755 28502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33115 every 8 connection(s)
I20260806 00:47:59.992578 28392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:47:59.994987 28504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:47:59.995764 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 28392
I20260806 00:47:59.995909 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:47:59.996222 30034 external_mini_cluster.cc:1428] Setting key d7c5a9bb112ae477f782da7b677ef26f
I20260806 00:48:00.001490 28504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208: Bootstrap starting.
I20260806 00:48:00.003957 28504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:00.004655 28504 log.cc:824] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:48:00 dist-test-slave-n326 krb5kdc[28373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:48:00.006512 28504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208: No bootstrap required, opened a new log
I20260806 00:48:00.010191 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:47:59.997405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58383 (local address 127.0.0.1:33115)
0806 00:47:59.997764 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 00:47:59.997782 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:47:59.997823 (+ 41us) server_negotiation.cc:407] Connection header received
0806 00:47:59.998627 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:47:59.998648 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:47:59.999052 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:47:59.999410 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:00.000060 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:00.001540 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:00.002408 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:00.003189 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:00.003343 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:00.005842 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:00.005847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:00.005858 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:00.005881 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:00.007591 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:00.008126 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:00.008130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:00.008134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:00.008237 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:00.008517 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:00.008519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:00.008520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:00.008846 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:00.008972 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:00.009290 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 00:48:00.009567 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":90,"threads_started":1}
I20260806 00:48:00.010358 28504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f03ea1dfe344d1996bb8665023de208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260806 00:48:00.010609 28504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:00.010667 28504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f03ea1dfe344d1996bb8665023de208, State: Initialized, Role: FOLLOWER
I20260806 00:48:00.011108 28504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f03ea1dfe344d1996bb8665023de208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260806 00:48:00.011278 28504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:00.011350 28504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:00.011482 28504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:00.012449 28504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f03ea1dfe344d1996bb8665023de208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260806 00:48:00.012761 28504 leader_election.cc:302] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f03ea1dfe344d1996bb8665023de208; no voters:
I20260806 00:48:00.013025 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:00.013067 28504 leader_election.cc:288] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:00.013168 28513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:00.013396 28513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [term 1 LEADER]: Becoming Leader. State: Replica: 9f03ea1dfe344d1996bb8665023de208, State: Running, Role: LEADER
I20260806 00:48:00.013763 28513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f03ea1dfe344d1996bb8665023de208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260806 00:48:00.014314 28504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:00.015420 28517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f03ea1dfe344d1996bb8665023de208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f03ea1dfe344d1996bb8665023de208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } } }
I20260806 00:48:00.015556 28517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:00.015872 28516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f03ea1dfe344d1996bb8665023de208. Latest consensus state: current_term: 1 leader_uuid: "9f03ea1dfe344d1996bb8665023de208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f03ea1dfe344d1996bb8665023de208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } } }
I20260806 00:48:00.015931 28533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:00.015961 28516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:00.018879 28533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:00.024286 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:00.016094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57731 (local address 127.0.0.1:33115)
0806 00:48:00.016257 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:00.016262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:00.016281 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:48:00.016317 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:00.016320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:00.016363 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:00.016438 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:00.017274 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:00.018011 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:00.019224 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:00.019585 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:00.019677 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:00.020962 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:00.020965 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:00.020968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:00.020990 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:00.022637 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:00.023101 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:00.023105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:00.023107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:00.023211 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:00.023526 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:00.023529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:00.023530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:00.023678 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:00.023809 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:00.024034 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 00:48:00.024140 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 00:48:00.025000 28533 catalog_manager.cc:1347] Generated new cluster ID: a1bd1e5b80424606a07ee5ca3cb7c97f
I20260806 00:48:00.025056 28533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:00.032847 28533 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:00.033677 28533 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:00.037945 28533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f03ea1dfe344d1996bb8665023de208: Generated new TSK 0
I20260806 00:48:00.038619 28533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:00.045255 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 28392
2026-08-06T00:48:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:00 dist-test-slave-n326 krb5kdc[28556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:00 dist-test-slave-n326 krb5kdc[28556](info): set up 2 sockets
Aug 06 00:48:00 dist-test-slave-n326 krb5kdc[28556](info): commencing operation
krb5kdc: starting...
W20260806 00:48:01.250943 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:01 dist-test-slave-n326 krb5kdc[28556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:48:01.371713 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:01.482892 28573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:01.483136 28573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:01.483180 28573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:01.487687 28573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:01.487739 28573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:01.487757 28573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:01.487797 28573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:01.487825 28573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:01.492959 28573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:01.493847 28573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:01.494761 28573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:01.502961 28584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:01.502974 28582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:01.503160 28573 server_base.cc:1034] running on GCE node
W20260806 00:48:01.502974 28588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:01.503803 28573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:01.504792 28573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:01.505983 28573 hybrid_clock.cc:648] HybridClock initialized: now 1785977281505971 us; error 37 us; skew 500 ppm
Aug 06 00:48:01 dist-test-slave-n326 krb5kdc[28556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977281, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:01.508769 28573 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:48:01.509572 28573 webserver.cc:460] Webserver started at http://127.0.0.1:41505/ using document root <none> and password file <none>
I20260806 00:48:01.510094 28573 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:01.510192 28573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:01.510414 28573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:01.512369 28573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77"
format_stamp: "Formatted at 2026-08-06 00:48:01 on dist-test-slave-n326"
server_key: "2ca2a671f196d52a476917eeee372748"
server_key_iv: "bce84af5a7813be09f8b5485dac74914"
server_key_version: "encryptionkey@0"
I20260806 00:48:01.512938 28573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77"
format_stamp: "Formatted at 2026-08-06 00:48:01 on dist-test-slave-n326"
server_key: "2ca2a671f196d52a476917eeee372748"
server_key_iv: "bce84af5a7813be09f8b5485dac74914"
server_key_version: "encryptionkey@0"
I20260806 00:48:01.516414 28573 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:48:01.518671 28602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:01.519822 28573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:01.519958 28573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77"
format_stamp: "Formatted at 2026-08-06 00:48:01 on dist-test-slave-n326"
server_key: "2ca2a671f196d52a476917eeee372748"
server_key_iv: "bce84af5a7813be09f8b5485dac74914"
server_key_version: "encryptionkey@0"
I20260806 00:48:01.520066 28573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:01.538061 28573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:01.540642 28573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:01.540827 28573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:01.547725 28573 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44653
I20260806 00:48:01.547761 28686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44653 every 8 connection(s)
I20260806 00:48:01.548571 28573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:01.551016 28687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:01.555991 28687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77: Bootstrap starting.
I20260806 00:48:01.557353 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 28573
I20260806 00:48:01.557511 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:01.557796 30034 external_mini_cluster.cc:1428] Setting key 06888c5bdbbcff006d433dc4c41d0d62
I20260806 00:48:01.558440 28687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:01.559319 28687 log.cc:824] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:01.561328 28687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77: No bootstrap required, opened a new log
I20260806 00:48:01.565443 28687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } }
I20260806 00:48:01.565687 28687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:01.565745 28687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d0cca7a75a84e7f9b7c3354cbe80b77, State: Initialized, Role: FOLLOWER
I20260806 00:48:01.566179 28687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } }
I20260806 00:48:01.566325 28687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:01.566386 28687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:01.566476 28687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:01.567093 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:01.558853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36867 (local address 127.0.0.1:44653)
0806 00:48:01.559312 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:01.559332 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:01.559375 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:48:01.560140 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:01.560161 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:01.560543 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:01.560873 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:01.561253 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:01.562697 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:01.563648 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:01.564399 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:01.564553 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:01.565245 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:01.565250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:01.565263 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:01.565714 (+ 451us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:01.565779 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:01.565812 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:01.565902 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:01.566001 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 00:48:01.566174 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":155,"threads_started":1}
I20260806 00:48:01.567451 28687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } }
I20260806 00:48:01.567757 28687 leader_election.cc:302] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d0cca7a75a84e7f9b7c3354cbe80b77; no voters:
I20260806 00:48:01.568049 28687 leader_election.cc:288] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:01.568125 28692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:01.568356 28692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [term 1 LEADER]: Becoming Leader. State: Replica: 2d0cca7a75a84e7f9b7c3354cbe80b77, State: Running, Role: LEADER
I20260806 00:48:01.568684 28692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } }
I20260806 00:48:01.569260 28687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:01.569796 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:01.570413 28699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } } }
I20260806 00:48:01.570447 28696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d0cca7a75a84e7f9b7c3354cbe80b77. Latest consensus state: current_term: 1 leader_uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0cca7a75a84e7f9b7c3354cbe80b77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } } }
I20260806 00:48:01.570544 28699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:01.570550 28696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:01.570964 28713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:01.573939 28713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:01.576712 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:01.572103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37205 (local address 127.0.0.1:44653)
0806 00:48:01.572202 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:01.572206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:01.572370 (+ 164us) server_negotiation.cc:407] Connection header received
0806 00:48:01.572538 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:01.572541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:01.572581 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:01.572667 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:01.573512 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:01.574110 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:01.575273 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:01.575505 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:01.575615 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:01.576297 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:01.576300 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:01.576303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:01.576313 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:01.576339 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:01.576354 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:01.576429 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:01.576515 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 00:48:01.576565 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 00:48:01.579618 28713 catalog_manager.cc:1347] Generated new cluster ID: a55ab8b5138948d1820f5cad2ed6ad30
I20260806 00:48:01.579692 28713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:01.588892 28713 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:01.589665 28713 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:01.602286 28713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d0cca7a75a84e7f9b7c3354cbe80b77: Generated new TSK 0
I20260806 00:48:01.602864 28713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:01.606669 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 28573
2026-08-06T00:48:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1562 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:01 dist-test-slave-n326 krb5kdc[28739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:01 dist-test-slave-n326 krb5kdc[28739](info): set up 2 sockets
Aug 06 00:48:01 dist-test-slave-n326 krb5kdc[28739](info): commencing operation
krb5kdc: starting...
W20260806 00:48:02.859160 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:02 dist-test-slave-n326 krb5kdc[28739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:48:02.977533 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:03.082765 28760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:03.083025 28760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:03.083112 28760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:03.086972 28760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:03.087054 28760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:03.087109 28760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:03.087147 28760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:03.087181 28760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:03.092125 28760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:03.093073 28760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:03.093992 28760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:03.101794 28766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:03.101795 28768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:03.101795 28771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:03.102200 28760 server_base.cc:1034] running on GCE node
I20260806 00:48:03.102695 28760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:03.103742 28760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:03.104946 28760 hybrid_clock.cc:648] HybridClock initialized: now 1785977283104920 us; error 40 us; skew 500 ppm
Aug 06 00:48:03 dist-test-slave-n326 krb5kdc[28739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977283, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:03.107628 28760 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:48:03.108422 28760 webserver.cc:460] Webserver started at http://127.0.0.1:39173/ using document root <none> and password file <none>
I20260806 00:48:03.108934 28760 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:03.109025 28760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:03.109246 28760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:03.111158 28760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8"
format_stamp: "Formatted at 2026-08-06 00:48:03 on dist-test-slave-n326"
server_key: "adc70ccfc95a61239804f98d76476a1b"
server_key_iv: "94086a2ebf9df8a3b9ff03df092506ef"
server_key_version: "encryptionkey@0"
I20260806 00:48:03.111713 28760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8"
format_stamp: "Formatted at 2026-08-06 00:48:03 on dist-test-slave-n326"
server_key: "adc70ccfc95a61239804f98d76476a1b"
server_key_iv: "94086a2ebf9df8a3b9ff03df092506ef"
server_key_version: "encryptionkey@0"
I20260806 00:48:03.115113 28760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 00:48:03.117331 28779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:03.118439 28760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:03.118577 28760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8"
format_stamp: "Formatted at 2026-08-06 00:48:03 on dist-test-slave-n326"
server_key: "adc70ccfc95a61239804f98d76476a1b"
server_key_iv: "94086a2ebf9df8a3b9ff03df092506ef"
server_key_version: "encryptionkey@0"
I20260806 00:48:03.118678 28760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:03.122800 28760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:03.125461 28760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:03.125644 28760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:03.132290 28760 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42333
I20260806 00:48:03.132337 28866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42333 every 8 connection(s)
I20260806 00:48:03.133057 28760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:03.135566 28867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:03.140307 28867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8: Bootstrap starting.
I20260806 00:48:03.142445 28867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:03.142586 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 28760
I20260806 00:48:03.142740 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:03.142997 30034 external_mini_cluster.cc:1428] Setting key 87ed26e5e3704b09b22ed3a75c6d4031
I20260806 00:48:03.143287 28867 log.cc:824] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:03.145459 28867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8: No bootstrap required, opened a new log
I20260806 00:48:03.148205 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:03.144067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39133 (local address 127.0.0.1:42333)
0806 00:48:03.144511 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:03.144527 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:03.144565 (+ 38us) server_negotiation.cc:407] Connection header received
0806 00:48:03.145358 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:03.145379 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:03.145718 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:03.146064 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:03.146787 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:03.146794 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:03.146810 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:03.146822 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:03.146886 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:03.146923 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:03.147029 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:03.147208 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 00:48:03.147351 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":135,"threads_started":1}
I20260806 00:48:03.149901 28867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } }
I20260806 00:48:03.150148 28867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:03.150210 28867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43a0be00fcb94c8bb23aa1a1f56aa6d8, State: Initialized, Role: FOLLOWER
I20260806 00:48:03.150688 28867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } }
I20260806 00:48:03.150868 28867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:03.150983 28867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:03.151124 28867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:03.151398 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:03.152047 28867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } }
I20260806 00:48:03.152371 28867 leader_election.cc:302] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43a0be00fcb94c8bb23aa1a1f56aa6d8; no voters:
I20260806 00:48:03.152663 28867 leader_election.cc:288] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:03.152781 28875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:03.152985 28875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [term 1 LEADER]: Becoming Leader. State: Replica: 43a0be00fcb94c8bb23aa1a1f56aa6d8, State: Running, Role: LEADER
I20260806 00:48:03.153339 28875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } }
I20260806 00:48:03.153985 28867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:03.154630 28883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43a0be00fcb94c8bb23aa1a1f56aa6d8. Latest consensus state: current_term: 1 leader_uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } } }
I20260806 00:48:03.154816 28883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:03.154975 28882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a0be00fcb94c8bb23aa1a1f56aa6d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } } }
I20260806 00:48:03.155151 28882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:03.155316 28898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:03.156013 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:03.153701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37915 (local address 127.0.0.1:42333)
0806 00:48:03.153795 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:03.153799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:03.153813 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:48:03.153881 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:03.153884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:03.153916 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:03.153965 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:03.155520 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:03.155524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:03.155526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:03.155538 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:03.155564 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:03.155581 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:03.155644 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:03.155846 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 00:48:03.155889 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 00:48:03.158883 28898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:03.163837 28898 catalog_manager.cc:1347] Generated new cluster ID: 85b6a17e54574e73a8ac67a0b096fd95
I20260806 00:48:03.163895 28898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:03.186604 28898 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:03.186894 28898 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:03.193892 28898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43a0be00fcb94c8bb23aa1a1f56aa6d8: Generated new TSK 0
I20260806 00:48:03.194468 28898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:03.198616 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 28760
2026-08-06T00:48:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:03 dist-test-slave-n326 krb5kdc[28915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:03 dist-test-slave-n326 krb5kdc[28915](info): set up 2 sockets
Aug 06 00:48:03 dist-test-slave-n326 krb5kdc[28915](info): commencing operation
krb5kdc: starting...
W20260806 00:48:04.404966 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:04 dist-test-slave-n326 krb5kdc[28915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:48:04.522149 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:04.627719 28933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:04.627933 28933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:04.628003 28933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:04.631915 28933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:04.631966 28933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:04.631985 28933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:04.632045 28933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:04.632076 28933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:04.636965 28933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:04.637837 28933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:04.638769 28933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:04.646679 28944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:04.646761 28939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:04.646679 28940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:04.647104 28933 server_base.cc:1034] running on GCE node
I20260806 00:48:04.647653 28933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:04.648883 28933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:04.650085 28933 hybrid_clock.cc:648] HybridClock initialized: now 1785977284650071 us; error 33 us; skew 500 ppm
Aug 06 00:48:04 dist-test-slave-n326 krb5kdc[28915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977284, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:04.658519 28933 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:48:04.659442 28933 webserver.cc:460] Webserver started at http://127.0.0.1:38875/ using document root <none> and password file <none>
I20260806 00:48:04.659968 28933 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:04.660058 28933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:04.660279 28933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:04.662264 28933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "f10f571c5cae47a4b51d8b0f303d7a25"
format_stamp: "Formatted at 2026-08-06 00:48:04 on dist-test-slave-n326"
server_key: "b1949f16014970f59a48b2e4fd5b6b5b"
server_key_iv: "9d55f447621dc763f970c3bd5ef16a38"
server_key_version: "encryptionkey@0"
I20260806 00:48:04.662948 28933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "f10f571c5cae47a4b51d8b0f303d7a25"
format_stamp: "Formatted at 2026-08-06 00:48:04 on dist-test-slave-n326"
server_key: "b1949f16014970f59a48b2e4fd5b6b5b"
server_key_iv: "9d55f447621dc763f970c3bd5ef16a38"
server_key_version: "encryptionkey@0"
I20260806 00:48:04.666306 28933 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:48:04.668608 28955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:04.669759 28933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:48:04.669899 28933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "f10f571c5cae47a4b51d8b0f303d7a25"
format_stamp: "Formatted at 2026-08-06 00:48:04 on dist-test-slave-n326"
server_key: "b1949f16014970f59a48b2e4fd5b6b5b"
server_key_iv: "9d55f447621dc763f970c3bd5ef16a38"
server_key_version: "encryptionkey@0"
I20260806 00:48:04.670004 28933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:04.680672 28933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:04.683146 28933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:04.683328 28933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:04.689870 28933 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44155
I20260806 00:48:04.689889 29031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44155 every 8 connection(s)
I20260806 00:48:04.690693 28933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:04.693691 29033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:04.697436 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 28933
I20260806 00:48:04.697580 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:04.697891 30034 external_mini_cluster.cc:1428] Setting key 9bbeb53c2b635adfb06298ced7714171
I20260806 00:48:04.699282 29033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25: Bootstrap starting.
I20260806 00:48:04.701473 29033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:04.702340 29033 log.cc:824] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:04.704291 29033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25: No bootstrap required, opened a new log
Aug 06 00:48:04 dist-test-slave-n326 krb5kdc[28915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:48:04.708731 29033 raft_consensus.cc:348] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10f571c5cae47a4b51d8b0f303d7a25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44155 } }
I20260806 00:48:04.708976 29033 raft_consensus.cc:374] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:04.709035 29033 raft_consensus.cc:729] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f10f571c5cae47a4b51d8b0f303d7a25, State: Initialized, Role: FOLLOWER
I20260806 00:48:04.709434 29033 consensus_queue.cc:260] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10f571c5cae47a4b51d8b0f303d7a25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44155 } }
I20260806 00:48:04.709573 29033 raft_consensus.cc:388] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:04.709658 29033 raft_consensus.cc:482] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:04.709782 29033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:04.710772 29033 raft_consensus.cc:504] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10f571c5cae47a4b51d8b0f303d7a25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44155 } }
I20260806 00:48:04.711186 29033 leader_election.cc:302] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f10f571c5cae47a4b51d8b0f303d7a25; no voters:
I20260806 00:48:04.711544 29033 leader_election.cc:288] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:04.711630 29041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:04.711841 29041 raft_consensus.cc:686] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [term 1 LEADER]: Becoming Leader. State: Replica: f10f571c5cae47a4b51d8b0f303d7a25, State: Running, Role: LEADER
I20260806 00:48:04.712179 29041 consensus_queue.cc:237] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10f571c5cae47a4b51d8b0f303d7a25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44155 } }
I20260806 00:48:04.712762 29033 sys_catalog.cc:564] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:04.713862 29042 sys_catalog.cc:455] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f10f571c5cae47a4b51d8b0f303d7a25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10f571c5cae47a4b51d8b0f303d7a25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44155 } } }
I20260806 00:48:04.714076 29042 sys_catalog.cc:458] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:04.714143 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:04.699411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:32949 (local address 127.0.0.1:44155)
0806 00:48:04.699851 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:04.699866 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:04.699909 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:48:04.700613 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:04.700629 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:04.700960 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:04.701238 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:04.702186 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:04.703344 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:04.704489 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:04.704971 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:04.705093 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:04.708059 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:04.708064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:04.708076 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:04.708103 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:04.710234 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:04.710884 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:04.710892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:04.710900 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:04.711092 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:04.711484 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:04.711487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:04.711488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:04.712075 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:04.712390 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:04.712682 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 00:48:04.713103 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":144,"threads_started":1}
I20260806 00:48:04.713450 29041 sys_catalog.cc:455] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f10f571c5cae47a4b51d8b0f303d7a25. Latest consensus state: current_term: 1 leader_uuid: "f10f571c5cae47a4b51d8b0f303d7a25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10f571c5cae47a4b51d8b0f303d7a25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44155 } } }
I20260806 00:48:04.715107 29041 sys_catalog.cc:458] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:04.717164 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 00:48:04.718238 29062 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:04.718326 29062 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:48:04.718430 29059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:04.719403 29059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:04.724815 29059 catalog_manager.cc:1347] Generated new cluster ID: bc7cb9ec2f8f4ee3a2507bebf3d39dcb
I20260806 00:48:04.724881 29059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:04.730962 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:04.719534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:48473 (local address 127.0.0.1:44155)
0806 00:48:04.719643 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:04.719648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:04.721634 (+ 1986us) server_negotiation.cc:407] Connection header received
0806 00:48:04.721775 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:04.721778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:04.721824 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:04.721904 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:04.722795 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:04.723477 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:04.724865 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:04.725282 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:04.725398 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:04.727326 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:04.727330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:04.727332 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:04.727355 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:04.729033 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:04.729628 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:04.729631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:04.729633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:04.729706 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:04.730088 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:04.730090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:04.730092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:04.730277 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:04.730435 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:04.730738 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 00:48:04.730821 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:48:04.749682 29059 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:04.750934 29059 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:04.757769 29059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f10f571c5cae47a4b51d8b0f303d7a25: Generated new TSK 0
I20260806 00:48:04.758572 29059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:04.766098 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 28933
W20260806 00:48:04.777953 28913 connection.cc:535] client connection to 127.0.0.1:44155 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44155): Connection reset by peer (error 104)
2026-08-06T00:48:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:04 dist-test-slave-n326 krb5kdc[29079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:04 dist-test-slave-n326 krb5kdc[29079](info): set up 2 sockets
Aug 06 00:48:04 dist-test-slave-n326 krb5kdc[29079](info): commencing operation
krb5kdc: starting...
W20260806 00:48:05.964588 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:06 dist-test-slave-n326 krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:48:06.082469 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:06.188616 29097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:06.188846 29097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:06.188885 29097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:06.192869 29097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:06.192919 29097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:06.192938 29097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:06.192955 29097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:06.192970 29097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:06.197901 29097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:06.198752 29097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:06.199992 29097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:06.210155 29108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:06.210147 29106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:06.210132 29103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:06.210379 29097 server_base.cc:1034] running on GCE node
I20260806 00:48:06.210944 29097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:06.211947 29097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:06.213105 29097 hybrid_clock.cc:648] HybridClock initialized: now 1785977286213088 us; error 29 us; skew 500 ppm
Aug 06 00:48:06 dist-test-slave-n326 krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977286, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:06.215749 29097 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:48:06.216519 29097 webserver.cc:460] Webserver started at http://127.0.0.1:43629/ using document root <none> and password file <none>
I20260806 00:48:06.217053 29097 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:06.217142 29097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:06.217370 29097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:06.219343 29097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "a31eee07a7174aa4a48ecfae6d095b9d"
format_stamp: "Formatted at 2026-08-06 00:48:06 on dist-test-slave-n326"
server_key: "5d3a194a79731b875de03bd459f04299"
server_key_iv: "1f8996e1e96a419cd2b4bf2bfe4fada3"
server_key_version: "encryptionkey@0"
I20260806 00:48:06.219875 29097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "a31eee07a7174aa4a48ecfae6d095b9d"
format_stamp: "Formatted at 2026-08-06 00:48:06 on dist-test-slave-n326"
server_key: "5d3a194a79731b875de03bd459f04299"
server_key_iv: "1f8996e1e96a419cd2b4bf2bfe4fada3"
server_key_version: "encryptionkey@0"
I20260806 00:48:06.223351 29097 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:48:06.225479 29118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:06.226569 29097 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:06.226701 29097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "a31eee07a7174aa4a48ecfae6d095b9d"
format_stamp: "Formatted at 2026-08-06 00:48:06 on dist-test-slave-n326"
server_key: "5d3a194a79731b875de03bd459f04299"
server_key_iv: "1f8996e1e96a419cd2b4bf2bfe4fada3"
server_key_version: "encryptionkey@0"
I20260806 00:48:06.226804 29097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:06.243202 29097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:06.245592 29097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:06.245777 29097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:06.252198 29097 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33243
I20260806 00:48:06.252226 29200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33243 every 8 connection(s)
I20260806 00:48:06.253062 29097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:06.255506 29201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:06.257953 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 29097
I20260806 00:48:06.258102 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:06.258464 30034 external_mini_cluster.cc:1428] Setting key 77103360535931ad77ca11fe73da68b3
I20260806 00:48:06.261243 29201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d: Bootstrap starting.
I20260806 00:48:06.263680 29201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:06.264446 29201 log.cc:824] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:06.266451 29201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d: No bootstrap required, opened a new log
I20260806 00:48:06.267038 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:06.259804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:57415 (local address 127.0.0.1:33243)
0806 00:48:06.260209 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:06.260221 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:06.260252 (+ 31us) server_negotiation.cc:407] Connection header received
0806 00:48:06.260819 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:06.260834 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:06.261142 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:06.261496 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:06.261837 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:06.262756 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:06.264016 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:06.264441 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:06.264604 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:06.265483 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:06.265488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:06.265499 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:06.265838 (+ 339us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:06.265885 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:06.265904 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:06.265999 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:06.266210 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 00:48:06.266352 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":122,"threads_started":1}
I20260806 00:48:06.269958 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:06.270296 29201 raft_consensus.cc:348] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31eee07a7174aa4a48ecfae6d095b9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } }
I20260806 00:48:06.270562 29201 raft_consensus.cc:374] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:06.270619 29201 raft_consensus.cc:729] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31eee07a7174aa4a48ecfae6d095b9d, State: Initialized, Role: FOLLOWER
I20260806 00:48:06.271047 29201 consensus_queue.cc:260] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31eee07a7174aa4a48ecfae6d095b9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } }
I20260806 00:48:06.271250 29201 raft_consensus.cc:388] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:06.271335 29201 raft_consensus.cc:482] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:06.271457 29201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:06.272536 29201 raft_consensus.cc:504] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31eee07a7174aa4a48ecfae6d095b9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } }
I20260806 00:48:06.272890 29201 leader_election.cc:302] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a31eee07a7174aa4a48ecfae6d095b9d; no voters:
I20260806 00:48:06.273252 29201 leader_election.cc:288] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:06.273343 29218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:06.273514 29218 raft_consensus.cc:686] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [term 1 LEADER]: Becoming Leader. State: Replica: a31eee07a7174aa4a48ecfae6d095b9d, State: Running, Role: LEADER
I20260806 00:48:06.273886 29218 consensus_queue.cc:237] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31eee07a7174aa4a48ecfae6d095b9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } }
I20260806 00:48:06.274425 29201 sys_catalog.cc:564] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:06.274994 29218 sys_catalog.cc:455] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a31eee07a7174aa4a48ecfae6d095b9d. Latest consensus state: current_term: 1 leader_uuid: "a31eee07a7174aa4a48ecfae6d095b9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31eee07a7174aa4a48ecfae6d095b9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } } }
I20260806 00:48:06.275774 29218 sys_catalog.cc:458] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:06.275646 29219 sys_catalog.cc:455] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a31eee07a7174aa4a48ecfae6d095b9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31eee07a7174aa4a48ecfae6d095b9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } } }
I20260806 00:48:06.276530 29219 sys_catalog.cc:458] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:06.276702 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:06.272203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:59611 (local address 127.0.0.1:33243)
0806 00:48:06.272300 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:06.272304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:06.272555 (+ 251us) server_negotiation.cc:407] Connection header received
0806 00:48:06.272679 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:06.272682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:06.272727 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:06.272802 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:06.273140 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:06.273833 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:06.274804 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:06.275175 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:06.275275 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:06.276217 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:06.276222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:06.276224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:06.276237 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:06.276263 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:06.276283 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:06.276367 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:06.276486 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 00:48:06.276514 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:48:06.277048 29230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:06.279829 29230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:06.284323 29230 catalog_manager.cc:1347] Generated new cluster ID: 24f387ca85274457ab98d45a9d713d1a
I20260806 00:48:06.284385 29230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:06.296114 29230 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:06.297161 29230 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:06.302794 29230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a31eee07a7174aa4a48ecfae6d095b9d: Generated new TSK 0
I20260806 00:48:06.303449 29230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:06.309958 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 29097
W20260806 00:48:06.321585 29076 connection.cc:535] client connection to 127.0.0.1:33243 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33243): Connection reset by peer (error 104)
2026-08-06T00:48:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:06 dist-test-slave-n326 krb5kdc[29247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:06 dist-test-slave-n326 krb5kdc[29247](info): set up 2 sockets
Aug 06 00:48:06 dist-test-slave-n326 krb5kdc[29247](info): commencing operation
krb5kdc: starting...
W20260806 00:48:07.517493 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:07 dist-test-slave-n326 krb5kdc[29247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:48:07.633842 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39597
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:07.739794 29265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:07.740065 29265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:07.740134 29265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:07.744024 29265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:07.744109 29265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:07.744148 29265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:07.744207 29265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:07.744247 29265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:07.749151 29265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:07.750077 29265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:07.751034 29265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:07.758895 29276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:07.758890 29279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:07.758886 29277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:07.759091 29265 server_base.cc:1034] running on GCE node
I20260806 00:48:07.759796 29265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:07.760761 29265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:07.761922 29265 hybrid_clock.cc:648] HybridClock initialized: now 1785977287761903 us; error 40 us; skew 500 ppm
Aug 06 00:48:07 dist-test-slave-n326 krb5kdc[29247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977287, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:07.764649 29265 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:48:07.765450 29265 webserver.cc:460] Webserver started at http://127.0.0.1:35133/ using document root <none> and password file <none>
I20260806 00:48:07.765950 29265 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:07.766009 29265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:07.766242 29265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:07.768129 29265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "ec2916de6da04af4ad26c31420aeee20"
format_stamp: "Formatted at 2026-08-06 00:48:07 on dist-test-slave-n326"
server_key: "bd8e940c8d2a99a563b8ee9c5db91d00"
server_key_iv: "06b83631a327b2658d8cfb2e0449c0b8"
server_key_version: "encryptionkey@0"
I20260806 00:48:07.768739 29265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "ec2916de6da04af4ad26c31420aeee20"
format_stamp: "Formatted at 2026-08-06 00:48:07 on dist-test-slave-n326"
server_key: "bd8e940c8d2a99a563b8ee9c5db91d00"
server_key_iv: "06b83631a327b2658d8cfb2e0449c0b8"
server_key_version: "encryptionkey@0"
I20260806 00:48:07.772089 29265 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 00:48:07.774139 29296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:07.775437 29265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:48:07.775576 29265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "ec2916de6da04af4ad26c31420aeee20"
format_stamp: "Formatted at 2026-08-06 00:48:07 on dist-test-slave-n326"
server_key: "bd8e940c8d2a99a563b8ee9c5db91d00"
server_key_iv: "06b83631a327b2658d8cfb2e0449c0b8"
server_key_version: "encryptionkey@0"
I20260806 00:48:07.775681 29265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:07.780006 29265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:07.783066 29265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:07.783304 29265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:07.790516 29265 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39597
I20260806 00:48:07.790586 29375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39597 every 8 connection(s)
I20260806 00:48:07.791539 29265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:07.794117 29376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:07.798985 29376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20: Bootstrap starting.
I20260806 00:48:07.799268 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 29265
I20260806 00:48:07.799433 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:07.799717 30034 external_mini_cluster.cc:1428] Setting key 97a4be26a700b38f4992c4b67793372a
I20260806 00:48:07.801441 29376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:07.802146 29376 log.cc:824] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:07.804129 29376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20: No bootstrap required, opened a new log
I20260806 00:48:07.804562 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:07.800871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:46861 (local address 127.0.0.1:39597)
0806 00:48:07.801277 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:07.801291 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:07.801336 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:48:07.801991 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:07.802009 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:07.802290 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:07.802647 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:07.803360 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:07.803364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:07.803375 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:07.803385 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:07.803432 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:07.803449 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:07.803568 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:07.803722 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 00:48:07.803877 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":124,"threads_started":1}
I20260806 00:48:07.807336 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:07.807991 29376 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2916de6da04af4ad26c31420aeee20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39597 } }
I20260806 00:48:07.808182 29376 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:07.808236 29376 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2916de6da04af4ad26c31420aeee20, State: Initialized, Role: FOLLOWER
I20260806 00:48:07.808694 29376 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2916de6da04af4ad26c31420aeee20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39597 } }
I20260806 00:48:07.808837 29376 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:07.808902 29376 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:07.809000 29376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:07.810101 29376 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2916de6da04af4ad26c31420aeee20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39597 } }
I20260806 00:48:07.810451 29376 leader_election.cc:302] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec2916de6da04af4ad26c31420aeee20; no voters:
I20260806 00:48:07.810787 29376 leader_election.cc:288] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:07.810931 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:07.809416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:45509 (local address 127.0.0.1:39597)
0806 00:48:07.809534 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:07.809537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:07.809701 (+ 164us) server_negotiation.cc:407] Connection header received
0806 00:48:07.809778 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:07.809781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:07.809811 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:07.809908 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:07.810494 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:07.810497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:07.810500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:07.810511 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:07.810536 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:07.810549 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:07.810650 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:07.810739 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 00:48:07.810776 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:48:07.811005 29404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:07.811265 29404 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [term 1 LEADER]: Becoming Leader. State: Replica: ec2916de6da04af4ad26c31420aeee20, State: Running, Role: LEADER
I20260806 00:48:07.811543 29404 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2916de6da04af4ad26c31420aeee20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39597 } }
I20260806 00:48:07.812004 29376 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:07.812600 29404 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec2916de6da04af4ad26c31420aeee20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2916de6da04af4ad26c31420aeee20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39597 } } }
I20260806 00:48:07.812739 29404 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:07.812969 29404 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec2916de6da04af4ad26c31420aeee20. Latest consensus state: current_term: 1 leader_uuid: "ec2916de6da04af4ad26c31420aeee20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2916de6da04af4ad26c31420aeee20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39597 } } }
I20260806 00:48:07.813197 29404 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:07.813519 29417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:07.816639 29417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:07.821955 29417 catalog_manager.cc:1347] Generated new cluster ID: d6764250cf0440ed8cd6af01032d37ca
I20260806 00:48:07.822029 29417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:07.834251 29417 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:07.834529 29417 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:07.838644 29417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec2916de6da04af4ad26c31420aeee20: Generated new TSK 0
I20260806 00:48:07.839263 29417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:07.845661 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 29265
2026-08-06T00:48:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:07 dist-test-slave-n326 krb5kdc[29434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:07 dist-test-slave-n326 krb5kdc[29434](info): set up 2 sockets
Aug 06 00:48:07 dist-test-slave-n326 krb5kdc[29434](info): commencing operation
krb5kdc: starting...
W20260806 00:48:09.043893 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:09 dist-test-slave-n326 krb5kdc[29434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:48:09.163813 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:09.270594 29448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:09.270836 29448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:09.270875 29448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:09.274742 29448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:09.274797 29448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:09.274816 29448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:09.274833 29448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:09.274848 29448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:09.279825 29448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:09.280673 29448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:09.281579 29448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:09.289619 29456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:09.289678 29461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:09.289757 29448 server_base.cc:1034] running on GCE node
W20260806 00:48:09.289746 29457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:09.290596 29448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:09.291666 29448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:09.292855 29448 hybrid_clock.cc:648] HybridClock initialized: now 1785977289292825 us; error 42 us; skew 500 ppm
Aug 06 00:48:09 dist-test-slave-n326 krb5kdc[29434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977289, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:09.295686 29448 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:48:09.296576 29448 webserver.cc:460] Webserver started at http://127.0.0.1:35665/ using document root <none> and password file <none>
I20260806 00:48:09.297113 29448 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:09.297209 29448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:09.297478 29448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:09.299496 29448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "d6ae555a4f7e4d3e94bc524ddeab665f"
format_stamp: "Formatted at 2026-08-06 00:48:09 on dist-test-slave-n326"
server_key: "1add411564c36bc800adf90abd55ef38"
server_key_iv: "ddf4f4c715f29b8c827abbf664086472"
server_key_version: "encryptionkey@0"
I20260806 00:48:09.300048 29448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "d6ae555a4f7e4d3e94bc524ddeab665f"
format_stamp: "Formatted at 2026-08-06 00:48:09 on dist-test-slave-n326"
server_key: "1add411564c36bc800adf90abd55ef38"
server_key_iv: "ddf4f4c715f29b8c827abbf664086472"
server_key_version: "encryptionkey@0"
I20260806 00:48:09.303638 29448 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:48:09.305922 29468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:09.307138 29448 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:09.307269 29448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "d6ae555a4f7e4d3e94bc524ddeab665f"
format_stamp: "Formatted at 2026-08-06 00:48:09 on dist-test-slave-n326"
server_key: "1add411564c36bc800adf90abd55ef38"
server_key_iv: "ddf4f4c715f29b8c827abbf664086472"
server_key_version: "encryptionkey@0"
I20260806 00:48:09.307384 29448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:09.325862 29448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:09.329082 29448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:09.329281 29448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:09.336304 29448 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33701
I20260806 00:48:09.336341 29549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33701 every 8 connection(s)
I20260806 00:48:09.337155 29448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:09.339643 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 29448
I20260806 00:48:09.339761 29550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:09.339788 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:09.340085 30034 external_mini_cluster.cc:1428] Setting key 30f76b3f4ee941e22a87d320977fc512
I20260806 00:48:09.345710 29550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f: Bootstrap starting.
I20260806 00:48:09.347841 29550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:09.348658 29550 log.cc:824] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:48:09 dist-test-slave-n326 krb5kdc[29434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:48:09.350795 29550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f: No bootstrap required, opened a new log
I20260806 00:48:09.354462 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:09.341438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:54323 (local address 127.0.0.1:33701)
0806 00:48:09.341887 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:09.341902 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:09.341935 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:48:09.342791 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:09.342809 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:09.343219 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:09.343541 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:09.344336 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:09.345287 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:09.346252 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:09.346724 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:09.346896 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:09.349797 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:09.349802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:09.349814 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:09.349840 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:09.351590 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:09.352132 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:09.352137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:09.352141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:09.352247 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:09.352607 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:09.352615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:09.352617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:09.352986 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:09.353227 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:09.353542 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 00:48:09.353749 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":132,"threads_started":1}
I20260806 00:48:09.354357 29550 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ae555a4f7e4d3e94bc524ddeab665f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } }
I20260806 00:48:09.354614 29550 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:09.354699 29550 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ae555a4f7e4d3e94bc524ddeab665f, State: Initialized, Role: FOLLOWER
I20260806 00:48:09.355224 29550 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ae555a4f7e4d3e94bc524ddeab665f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } }
I20260806 00:48:09.355381 29550 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:09.355473 29550 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:09.355599 29550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:09.356739 29550 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ae555a4f7e4d3e94bc524ddeab665f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } }
I20260806 00:48:09.357120 29550 leader_election.cc:302] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6ae555a4f7e4d3e94bc524ddeab665f; no voters:
I20260806 00:48:09.357471 29550 leader_election.cc:288] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:09.357578 29556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:09.357800 29556 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [term 1 LEADER]: Becoming Leader. State: Replica: d6ae555a4f7e4d3e94bc524ddeab665f, State: Running, Role: LEADER
I20260806 00:48:09.357841 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:09.358196 29556 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ae555a4f7e4d3e94bc524ddeab665f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } }
I20260806 00:48:09.358815 29550 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:09.359593 29556 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6ae555a4f7e4d3e94bc524ddeab665f. Latest consensus state: current_term: 1 leader_uuid: "d6ae555a4f7e4d3e94bc524ddeab665f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ae555a4f7e4d3e94bc524ddeab665f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } } }
I20260806 00:48:09.359768 29556 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:09.360150 29557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6ae555a4f7e4d3e94bc524ddeab665f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ae555a4f7e4d3e94bc524ddeab665f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } } }
I20260806 00:48:09.360255 29557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:09.360229 29570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:09.363729 29570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:09.369236 29570 catalog_manager.cc:1347] Generated new cluster ID: 52963391fa904d9db975f0d063babb3c
I20260806 00:48:09.369295 29570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:09.373517 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:09.361608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:44235 (local address 127.0.0.1:33701)
0806 00:48:09.361704 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:09.361710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:09.364427 (+ 2717us) server_negotiation.cc:407] Connection header received
0806 00:48:09.364582 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:09.364588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:09.364637 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:09.364724 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:09.365607 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:09.366370 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:09.367644 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:09.367945 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:09.368077 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:09.369788 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:09.369792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:09.369795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:09.369820 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:09.371502 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:09.372178 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:09.372182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:09.372184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:09.372270 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:09.372644 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:09.372647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:09.372648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:09.372833 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:09.372992 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:09.373276 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 00:48:09.373370 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 00:48:09.384346 29570 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:09.385248 29570 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:09.394901 29570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6ae555a4f7e4d3e94bc524ddeab665f: Generated new TSK 0
I20260806 00:48:09.395663 29570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:09.402210 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 29448
W20260806 00:48:09.412024 29432 connection.cc:535] client connection to 127.0.0.1:33701 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33701): Connection reset by peer (error 104)
2026-08-06T00:48:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1557 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:09 dist-test-slave-n326 krb5kdc[29595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:09 dist-test-slave-n326 krb5kdc[29595](info): set up 2 sockets
Aug 06 00:48:09 dist-test-slave-n326 krb5kdc[29595](info): commencing operation
krb5kdc: starting...
W20260806 00:48:10.610266 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:10 dist-test-slave-n326 krb5kdc[29595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:48:10.730322 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35325
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:10.836431 29613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:10.836700 29613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:10.836776 29613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:10.840711 29613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:10.840796 29613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:10.840834 29613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:10.840890 29613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:10.840930 29613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:10.845918 29613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35325
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:10.846817 29613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:10.847744 29613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:10.855757 29624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:10.855751 29619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:10.855751 29620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:10.855931 29613 server_base.cc:1034] running on GCE node
I20260806 00:48:10.856593 29613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:10.857499 29613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:10.858661 29613 hybrid_clock.cc:648] HybridClock initialized: now 1785977290858646 us; error 29 us; skew 500 ppm
Aug 06 00:48:10 dist-test-slave-n326 krb5kdc[29595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977290, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:10.867213 29613 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:48:10.868075 29613 webserver.cc:460] Webserver started at http://127.0.0.1:44669/ using document root <none> and password file <none>
I20260806 00:48:10.868628 29613 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:10.868727 29613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:10.868963 29613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:10.870873 29613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "bfd311c8ab864d90a2bf7e6eae0acde4"
format_stamp: "Formatted at 2026-08-06 00:48:10 on dist-test-slave-n326"
server_key: "c05333e71ad91bcf5db8a614df11d250"
server_key_iv: "c8a7b6568649b25cfe7ad3d4504edaed"
server_key_version: "encryptionkey@0"
I20260806 00:48:10.871472 29613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "bfd311c8ab864d90a2bf7e6eae0acde4"
format_stamp: "Formatted at 2026-08-06 00:48:10 on dist-test-slave-n326"
server_key: "c05333e71ad91bcf5db8a614df11d250"
server_key_iv: "c8a7b6568649b25cfe7ad3d4504edaed"
server_key_version: "encryptionkey@0"
I20260806 00:48:10.874922 29613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:48:10.877207 29634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:10.878346 29613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:10.878501 29613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "bfd311c8ab864d90a2bf7e6eae0acde4"
format_stamp: "Formatted at 2026-08-06 00:48:10 on dist-test-slave-n326"
server_key: "c05333e71ad91bcf5db8a614df11d250"
server_key_iv: "c8a7b6568649b25cfe7ad3d4504edaed"
server_key_version: "encryptionkey@0"
I20260806 00:48:10.878611 29613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:10.898432 29613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:10.901204 29613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:10.901407 29613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:10.908646 29613 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35325
I20260806 00:48:10.908676 29714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35325 every 8 connection(s)
I20260806 00:48:10.909526 29613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:10.912039 29715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:10.915817 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 29613
I20260806 00:48:10.915966 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:10.916260 30034 external_mini_cluster.cc:1428] Setting key ea7919cd30f331e577928c3ef53bf87a
I20260806 00:48:10.917348 29715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4: Bootstrap starting.
I20260806 00:48:10.919509 29715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:10.920225 29715 log.cc:824] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:10.921926 29715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4: No bootstrap required, opened a new log
I20260806 00:48:10.924733 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:10.917586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:34681 (local address 127.0.0.1:35325)
0806 00:48:10.917979 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:10.917993 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:10.918032 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:48:10.918619 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:10.918634 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:10.918930 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:10.919334 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:10.919731 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:10.920728 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:10.921642 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:10.922157 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:10.922328 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:10.923235 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:10.923241 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:10.923262 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:10.923608 (+ 346us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:10.923656 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:10.923678 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:10.923816 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:10.923992 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 00:48:10.924125 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":121,"threads_started":1}
I20260806 00:48:10.925560 29715 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd311c8ab864d90a2bf7e6eae0acde4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } }
I20260806 00:48:10.925804 29715 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:10.925869 29715 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd311c8ab864d90a2bf7e6eae0acde4, State: Initialized, Role: FOLLOWER
I20260806 00:48:10.926276 29715 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd311c8ab864d90a2bf7e6eae0acde4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } }
I20260806 00:48:10.926457 29715 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:10.926533 29715 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:10.926630 29715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:10.927502 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:10.927637 29715 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd311c8ab864d90a2bf7e6eae0acde4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } }
I20260806 00:48:10.927942 29715 leader_election.cc:302] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfd311c8ab864d90a2bf7e6eae0acde4; no voters:
I20260806 00:48:10.928270 29715 leader_election.cc:288] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:10.928363 29723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:10.928570 29723 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [term 1 LEADER]: Becoming Leader. State: Replica: bfd311c8ab864d90a2bf7e6eae0acde4, State: Running, Role: LEADER
I20260806 00:48:10.928903 29723 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd311c8ab864d90a2bf7e6eae0acde4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } }
I20260806 00:48:10.929494 29715 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:10.930105 29723 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd311c8ab864d90a2bf7e6eae0acde4. Latest consensus state: current_term: 1 leader_uuid: "bfd311c8ab864d90a2bf7e6eae0acde4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd311c8ab864d90a2bf7e6eae0acde4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } } }
I20260806 00:48:10.930239 29723 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:10.930527 29725 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfd311c8ab864d90a2bf7e6eae0acde4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd311c8ab864d90a2bf7e6eae0acde4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } } }
I20260806 00:48:10.930613 29725 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:10.930652 29740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:10.933606 29740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:10.935046 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:10.929686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:44587 (local address 127.0.0.1:35325)
0806 00:48:10.929777 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:10.929781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:10.930016 (+ 235us) server_negotiation.cc:407] Connection header received
0806 00:48:10.930885 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:10.930888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:10.930929 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:10.930974 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:10.931411 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:10.932106 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:10.933213 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:10.933536 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:10.933651 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:10.934519 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:10.934523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:10.934525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:10.934538 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:10.934563 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:10.934575 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:10.934644 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:10.934882 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 00:48:10.934914 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 00:48:10.939025 29740 catalog_manager.cc:1347] Generated new cluster ID: 55f679a51d234d3fb084862864e6bde3
I20260806 00:48:10.939098 29740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:10.975219 29740 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:10.976263 29740 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:10.989347 29740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfd311c8ab864d90a2bf7e6eae0acde4: Generated new TSK 0
I20260806 00:48:10.989969 29740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:10.998662 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 29613
W20260806 00:48:11.007021 29592 connection.cc:535] client connection to 127.0.0.1:35325 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35325): Connection reset by peer (error 104)
2026-08-06T00:48:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:11 dist-test-slave-n326 krb5kdc[29771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:11 dist-test-slave-n326 krb5kdc[29771](info): set up 2 sockets
Aug 06 00:48:11 dist-test-slave-n326 krb5kdc[29771](info): commencing operation
krb5kdc: starting...
W20260806 00:48:12.214663 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:12 dist-test-slave-n326 krb5kdc[29771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:48:12.340319 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:12.450567 29790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:12.450834 29790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:12.450906 29790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:12.454782 29790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:12.454864 29790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:12.454902 29790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:12.454936 29790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:12.454965 29790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:12.460067 29790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40881
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:12.461005 29790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:12.461987 29790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:12.470314 29800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:12.470494 29790 server_base.cc:1034] running on GCE node
W20260806 00:48:12.470358 29797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:12.470333 29795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:12.471263 29790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:12.472208 29790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:12.473367 29790 hybrid_clock.cc:648] HybridClock initialized: now 1785977292473349 us; error 29 us; skew 500 ppm
Aug 06 00:48:12 dist-test-slave-n326 krb5kdc[29771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977292, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:12.476166 29790 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:48:12.476915 29790 webserver.cc:460] Webserver started at http://127.0.0.1:38845/ using document root <none> and password file <none>
I20260806 00:48:12.477406 29790 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:12.477463 29790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:12.477612 29790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:12.479506 29790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "efb744014af849b5ab80dd125cbe56c2"
format_stamp: "Formatted at 2026-08-06 00:48:12 on dist-test-slave-n326"
server_key: "8dd24e40a731dc10f47055f6cd3d455b"
server_key_iv: "9971139be105a743378742e31ec9a4be"
server_key_version: "encryptionkey@0"
I20260806 00:48:12.479988 29790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "efb744014af849b5ab80dd125cbe56c2"
format_stamp: "Formatted at 2026-08-06 00:48:12 on dist-test-slave-n326"
server_key: "8dd24e40a731dc10f47055f6cd3d455b"
server_key_iv: "9971139be105a743378742e31ec9a4be"
server_key_version: "encryptionkey@0"
I20260806 00:48:12.483436 29790 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:48:12.485673 29812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:12.486765 29790 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:12.486899 29790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "efb744014af849b5ab80dd125cbe56c2"
format_stamp: "Formatted at 2026-08-06 00:48:12 on dist-test-slave-n326"
server_key: "8dd24e40a731dc10f47055f6cd3d455b"
server_key_iv: "9971139be105a743378742e31ec9a4be"
server_key_version: "encryptionkey@0"
I20260806 00:48:12.486994 29790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:12.491264 29790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:12.494221 29790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:12.494406 29790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:12.501127 29790 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40881
I20260806 00:48:12.501149 29888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40881 every 8 connection(s)
I20260806 00:48:12.501957 29790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:12.504720 29891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:12.505784 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 29790
I20260806 00:48:12.505975 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:12.506255 30034 external_mini_cluster.cc:1428] Setting key a7f8646a8d1bf63ade5a7fdce7176f71
I20260806 00:48:12.511382 29891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2: Bootstrap starting.
I20260806 00:48:12.511523 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:12.507484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:59763 (local address 127.0.0.1:40881)
0806 00:48:12.507889 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:12.507909 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:12.508179 (+ 270us) server_negotiation.cc:407] Connection header received
0806 00:48:12.508910 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:12.508925 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:12.509196 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:12.509506 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:12.510195 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:12.510200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:12.510211 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:12.510220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:12.510265 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:12.510281 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:12.510443 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:12.510509 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 00:48:12.510662 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":128,"threads_started":1}
I20260806 00:48:12.513595 29891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:12.514122 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:12.514391 29891 log.cc:824] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:12.516665 29891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2: No bootstrap required, opened a new log
I20260806 00:48:12.518105 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:12.516549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:57315 (local address 127.0.0.1:40881)
0806 00:48:12.516659 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:12.516664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:12.516843 (+ 179us) server_negotiation.cc:407] Connection header received
0806 00:48:12.516990 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:12.516992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:12.517021 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:12.517079 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:12.517704 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:12.517707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:12.517708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:12.517718 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:12.517739 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:12.517749 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:12.517814 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:12.517932 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 00:48:12.517963 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:48:12.520570 29891 raft_consensus.cc:348] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb744014af849b5ab80dd125cbe56c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260806 00:48:12.520766 29891 raft_consensus.cc:374] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:12.520856 29891 raft_consensus.cc:729] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efb744014af849b5ab80dd125cbe56c2, State: Initialized, Role: FOLLOWER
I20260806 00:48:12.521243 29891 consensus_queue.cc:260] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb744014af849b5ab80dd125cbe56c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260806 00:48:12.521392 29891 raft_consensus.cc:388] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:12.521485 29891 raft_consensus.cc:482] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:12.521581 29891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:12.522372 29891 raft_consensus.cc:504] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb744014af849b5ab80dd125cbe56c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260806 00:48:12.522694 29891 leader_election.cc:302] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efb744014af849b5ab80dd125cbe56c2; no voters:
I20260806 00:48:12.522992 29891 leader_election.cc:288] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:12.523157 29907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:12.523393 29907 raft_consensus.cc:686] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [term 1 LEADER]: Becoming Leader. State: Replica: efb744014af849b5ab80dd125cbe56c2, State: Running, Role: LEADER
I20260806 00:48:12.523715 29907 consensus_queue.cc:237] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb744014af849b5ab80dd125cbe56c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260806 00:48:12.524179 29891 sys_catalog.cc:564] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:12.525370 29909 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efb744014af849b5ab80dd125cbe56c2. Latest consensus state: current_term: 1 leader_uuid: "efb744014af849b5ab80dd125cbe56c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb744014af849b5ab80dd125cbe56c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } } }
I20260806 00:48:12.525507 29909 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:12.525847 29908 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efb744014af849b5ab80dd125cbe56c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb744014af849b5ab80dd125cbe56c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } } }
I20260806 00:48:12.525930 29921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:12.526034 29908 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:12.528626 29921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:12.533603 29921 catalog_manager.cc:1347] Generated new cluster ID: 29f49c934fdc4cf39b29fc9c97aadadd
I20260806 00:48:12.533674 29921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:12.540491 29921 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:12.540742 29921 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:12.546185 29921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efb744014af849b5ab80dd125cbe56c2: Generated new TSK 0
I20260806 00:48:12.546737 29921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:12.556185 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 29790
2026-08-06T00:48:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1567 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:12 dist-test-slave-n326 krb5kdc[29941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:12 dist-test-slave-n326 krb5kdc[29941](info): set up 2 sockets
Aug 06 00:48:12 dist-test-slave-n326 krb5kdc[29941](info): commencing operation
krb5kdc: starting...
W20260806 00:48:13.775251 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:13 dist-test-slave-n326 krb5kdc[29941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:48:13.892210 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40319
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:40319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:13.999379 29959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:13.999639 29959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:13.999717 29959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:14.003607 29959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:14.003690 29959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:14.003728 29959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:14.003786 29959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:14.003827 29959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:14.008790 29959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:40319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:40319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:14.009743 29959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:14.010694 29959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:14.018805 29967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:14.018852 29964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:14.018975 29959 server_base.cc:1034] running on GCE node
W20260806 00:48:14.018824 29965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:14.019671 29959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:14.020720 29959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:14.021903 29959 hybrid_clock.cc:648] HybridClock initialized: now 1785977294021884 us; error 30 us; skew 500 ppm
Aug 06 00:48:14 dist-test-slave-n326 krb5kdc[29941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977294, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:14.024638 29959 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:48:14.025475 29959 webserver.cc:460] Webserver started at http://127.29.84.190:32789/ using document root <none> and password file <none>
I20260806 00:48:14.025949 29959 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:14.026005 29959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:14.026163 29959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:14.028028 29959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "8bb2cccedf314ef09829e84aebd99684"
format_stamp: "Formatted at 2026-08-06 00:48:14 on dist-test-slave-n326"
server_key: "8b2a6ad97b953dc75bb02d4877fa9b8c"
server_key_iv: "6bdf1176c9aa9a10e0e0b13b14f25e7e"
server_key_version: "encryptionkey@0"
I20260806 00:48:14.028501 29959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "8bb2cccedf314ef09829e84aebd99684"
format_stamp: "Formatted at 2026-08-06 00:48:14 on dist-test-slave-n326"
server_key: "8b2a6ad97b953dc75bb02d4877fa9b8c"
server_key_iv: "6bdf1176c9aa9a10e0e0b13b14f25e7e"
server_key_version: "encryptionkey@0"
I20260806 00:48:14.031857 29959 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:48:14.033927 29977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:14.035005 29959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:14.035180 29959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "8bb2cccedf314ef09829e84aebd99684"
format_stamp: "Formatted at 2026-08-06 00:48:14 on dist-test-slave-n326"
server_key: "8b2a6ad97b953dc75bb02d4877fa9b8c"
server_key_iv: "6bdf1176c9aa9a10e0e0b13b14f25e7e"
server_key_version: "encryptionkey@0"
I20260806 00:48:14.035274 29959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:14.051018 29959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:14.053722 29959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:14.053905 29959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:14.060854 29959 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:40319
I20260806 00:48:14.060890 30073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:40319 every 8 connection(s)
I20260806 00:48:14.061738 29959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:14.064276 30075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:14.067155 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 29959
I20260806 00:48:14.067287 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:14.067557 30034 external_mini_cluster.cc:1428] Setting key a10040f351bf17ed719a07625dd0b1a6
I20260806 00:48:14.069801 30075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684: Bootstrap starting.
I20260806 00:48:14.071877 30075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:14.072576 30075 log.cc:824] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:14.074242 30075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684: No bootstrap required, opened a new log
Aug 06 00:48:14 dist-test-slave-n326 krb5kdc[29941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:48:14.077819 30075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb2cccedf314ef09829e84aebd99684" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40319 } }
I20260806 00:48:14.078018 30075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:14.078076 30075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bb2cccedf314ef09829e84aebd99684, State: Initialized, Role: FOLLOWER
I20260806 00:48:14.078518 30075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb2cccedf314ef09829e84aebd99684" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40319 } }
I20260806 00:48:14.078658 30075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:14.078727 30075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:14.078828 30075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:14.079885 30075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb2cccedf314ef09829e84aebd99684" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40319 } }
I20260806 00:48:14.080253 30075 leader_election.cc:302] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bb2cccedf314ef09829e84aebd99684; no voters:
I20260806 00:48:14.080602 30075 leader_election.cc:288] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:14.080703 30083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:14.081182 30083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [term 1 LEADER]: Becoming Leader. State: Replica: 8bb2cccedf314ef09829e84aebd99684, State: Running, Role: LEADER
I20260806 00:48:14.081537 30083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb2cccedf314ef09829e84aebd99684" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40319 } }
I20260806 00:48:14.081854 30075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:14.082136 30080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:14.068858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46729 (local address 127.29.84.190:40319)
0806 00:48:14.069270 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:14.069284 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:14.069319 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:48:14.069949 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:14.069971 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:14.070296 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:14.070629 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:14.071478 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:14.072479 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:14.073392 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:14.073849 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:14.074018 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:14.076729 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:14.076734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:14.076745 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:14.076770 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:14.078547 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:14.079172 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:14.079177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:14.079182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:14.079309 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:14.079684 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:14.079687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:14.079688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:14.080092 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:14.080253 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:14.081004 (+ 751us) server_negotiation.cc:299] Negotiation successful
0806 00:48:14.081206 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":126,"threads_started":1}
I20260806 00:48:14.083189 30086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bb2cccedf314ef09829e84aebd99684. Latest consensus state: current_term: 1 leader_uuid: "8bb2cccedf314ef09829e84aebd99684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb2cccedf314ef09829e84aebd99684" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40319 } } }
I20260806 00:48:14.083319 30086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:14.083639 30085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bb2cccedf314ef09829e84aebd99684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb2cccedf314ef09829e84aebd99684" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 40319 } } }
I20260806 00:48:14.083726 30085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:14.083770 30096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:14.085295 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:14.086745 30096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:14.092401 30096 catalog_manager.cc:1347] Generated new cluster ID: f9cbde96d0f94086beed8bb29a337009
I20260806 00:48:14.092517 30096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:14.095597 30080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:14.087681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46971 (local address 127.29.84.190:40319)
0806 00:48:14.087778 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:14.087782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:14.088033 (+ 251us) server_negotiation.cc:407] Connection header received
0806 00:48:14.088122 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:14.088125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:14.088165 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:14.088244 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:14.088939 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:14.089452 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:14.090261 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:14.090545 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:14.090665 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:14.092318 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:14.092321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:14.092323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:14.092345 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:14.093846 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:14.094312 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:14.094315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:14.094317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:14.094401 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:14.094732 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:14.094735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:14.094736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:14.094893 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:14.095039 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:14.095334 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 00:48:14.095419 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 00:48:14.102473 30096 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:14.103332 30096 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:14.109073 30096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bb2cccedf314ef09829e84aebd99684: Generated new TSK 0
I20260806 00:48:14.109627 30096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:14.116418 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 29959
W20260806 00:48:14.127712 29934 connection.cc:535] client connection to 127.29.84.190:40319 recv error: Network error: failed to read from TLS socket (remote: 127.29.84.190:40319): Connection reset by peer (error 104)
2026-08-06T00:48:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:14 dist-test-slave-n326 krb5kdc[30119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:14 dist-test-slave-n326 krb5kdc[30119](info): set up 2 sockets
Aug 06 00:48:14 dist-test-slave-n326 krb5kdc[30119](info): commencing operation
krb5kdc: starting...
W20260806 00:48:15.334793 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:15 dist-test-slave-n326 krb5kdc[30119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:48:15.453504 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43305
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:46091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:15.561022 30139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:15.561278 30139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:15.561348 30139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:15.565183 30139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:15.565263 30139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:15.565300 30139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:15.565335 30139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:15.565369 30139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:15.570312 30139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:46091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:43305
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:15.571251 30139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:15.572268 30139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:15.580229 30150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:15.580248 30154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:15.580299 30151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:15.580440 30139 server_base.cc:1034] running on GCE node
I20260806 00:48:15.581136 30139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:15.582145 30139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:15.583331 30139 hybrid_clock.cc:648] HybridClock initialized: now 1785977295583314 us; error 33 us; skew 500 ppm
Aug 06 00:48:15 dist-test-slave-n326 krb5kdc[30119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:15.585958 30139 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:48:15.586789 30139 webserver.cc:460] Webserver started at http://127.29.84.190:39505/ using document root <none> and password file <none>
I20260806 00:48:15.587383 30139 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:15.587471 30139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:15.587702 30139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:15.589605 30139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "16d31e47f783420face457b3a556ae6f"
format_stamp: "Formatted at 2026-08-06 00:48:15 on dist-test-slave-n326"
server_key: "c6b4520d0df63e4495541209a9955cf5"
server_key_iv: "c2a0855e0203c5d22563f6fc6a416de5"
server_key_version: "encryptionkey@0"
I20260806 00:48:15.590144 30139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "16d31e47f783420face457b3a556ae6f"
format_stamp: "Formatted at 2026-08-06 00:48:15 on dist-test-slave-n326"
server_key: "c6b4520d0df63e4495541209a9955cf5"
server_key_iv: "c2a0855e0203c5d22563f6fc6a416de5"
server_key_version: "encryptionkey@0"
I20260806 00:48:15.593636 30139 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 00:48:15.595917 30162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:15.596999 30139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 00:48:15.597136 30139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "16d31e47f783420face457b3a556ae6f"
format_stamp: "Formatted at 2026-08-06 00:48:15 on dist-test-slave-n326"
server_key: "c6b4520d0df63e4495541209a9955cf5"
server_key_iv: "c2a0855e0203c5d22563f6fc6a416de5"
server_key_version: "encryptionkey@0"
I20260806 00:48:15.597251 30139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:15.609439 30139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:15.612190 30139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:15.612387 30139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:15.619163 30139 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:43305
I20260806 00:48:15.619181 30242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:43305 every 8 connection(s)
I20260806 00:48:15.620050 30139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:15.622609 30244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:15.627918 30244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f: Bootstrap starting.
I20260806 00:48:15.628626 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30139
I20260806 00:48:15.628777 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:15.629068 30034 external_mini_cluster.cc:1428] Setting key ec9e782727dc146ebf7e382383bf76df
I20260806 00:48:15.630666 30244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:15.631577 30244 log.cc:824] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:15.633736 30244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f: No bootstrap required, opened a new log
I20260806 00:48:15.637907 30244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d31e47f783420face457b3a556ae6f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43305 } }
I20260806 00:48:15.638163 30244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:15.638221 30244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d31e47f783420face457b3a556ae6f, State: Initialized, Role: FOLLOWER
I20260806 00:48:15.638724 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:15.630183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41767 (local address 127.29.84.190:43305)
0806 00:48:15.630663 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:15.630685 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:15.630730 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:48:15.631614 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:15.631637 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:15.632059 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:15.632383 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:15.632803 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:15.634204 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:15.635097 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:15.635803 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:15.635963 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:15.636776 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:15.636782 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:15.636797 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:15.637271 (+ 474us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:15.637337 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:15.637365 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:15.637513 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:15.637681 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 00:48:15.637802 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":136,"threads_started":1}
I20260806 00:48:15.638675 30244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d31e47f783420face457b3a556ae6f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43305 } }
I20260806 00:48:15.638823 30244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:15.638887 30244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:15.639011 30244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:15.639971 30244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d31e47f783420face457b3a556ae6f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43305 } }
I20260806 00:48:15.640307 30244 leader_election.cc:302] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16d31e47f783420face457b3a556ae6f; no voters:
I20260806 00:48:15.640635 30244 leader_election.cc:288] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:15.640725 30250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:15.640908 30250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [term 1 LEADER]: Becoming Leader. State: Replica: 16d31e47f783420face457b3a556ae6f, State: Running, Role: LEADER
I20260806 00:48:15.641240 30250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d31e47f783420face457b3a556ae6f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43305 } }
I20260806 00:48:15.641736 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:15.641855 30244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:15.642908 30252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16d31e47f783420face457b3a556ae6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d31e47f783420face457b3a556ae6f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43305 } } }
I20260806 00:48:15.643049 30252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:15.643392 30255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16d31e47f783420face457b3a556ae6f. Latest consensus state: current_term: 1 leader_uuid: "16d31e47f783420face457b3a556ae6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d31e47f783420face457b3a556ae6f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43305 } } }
I20260806 00:48:15.643481 30255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f [sys.catalog]: This master's current role is: LEADER
W20260806 00:48:15.646833 30279 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:15.646935 30279 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:48:15.647027 30271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:15.648023 30271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:15.648751 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:15.643992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39233 (local address 127.29.84.190:43305)
0806 00:48:15.644104 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:15.644108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:15.644578 (+ 470us) server_negotiation.cc:407] Connection header received
0806 00:48:15.644666 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:15.644669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:15.644708 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:15.644790 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:15.645093 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:15.645831 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:15.646956 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:15.647311 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:15.647422 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:15.648283 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:15.648287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:15.648290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:15.648301 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:15.648326 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:15.648339 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:15.648432 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:15.648607 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 00:48:15.648637 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:48:15.653712 30271 catalog_manager.cc:1347] Generated new cluster ID: c2d2e7f08f53465f9209cc43215b4e7f
I20260806 00:48:15.653784 30271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:15.662771 30271 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:15.663619 30271 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:15.669184 30271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16d31e47f783420face457b3a556ae6f: Generated new TSK 0
I20260806 00:48:15.669746 30271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:15.675578 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30139
W20260806 00:48:15.692309 30116 connection.cc:535] client connection to 127.29.84.190:43305 recv error: Network error: failed to read from TLS socket (remote: 127.29.84.190:43305): Connection reset by peer (error 104)
2026-08-06T00:48:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1564 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:15 dist-test-slave-n326 krb5kdc[30289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:15 dist-test-slave-n326 krb5kdc[30289](info): set up 2 sockets
Aug 06 00:48:15 dist-test-slave-n326 krb5kdc[30289](info): commencing operation
krb5kdc: starting...
W20260806 00:48:16.887393 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:16 dist-test-slave-n326 krb5kdc[30289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:48:17.007139 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43477
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:43477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:17.113353 30308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:17.113581 30308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:17.113620 30308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:17.117460 30308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:17.117509 30308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:17.117528 30308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:17.117544 30308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:17.117588 30308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:17.122555 30308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:43477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:17.123476 30308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:17.124383 30308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:17.132153 30313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:17.132161 30316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:17.132169 30314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:17.132372 30308 server_base.cc:1034] running on GCE node
I20260806 00:48:17.133087 30308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:17.134027 30308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:17.135212 30308 hybrid_clock.cc:648] HybridClock initialized: now 1785977297135194 us; error 28 us; skew 500 ppm
Aug 06 00:48:17 dist-test-slave-n326 krb5kdc[30289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977297, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:17.142004 30308 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:48:17.142817 30308 webserver.cc:460] Webserver started at http://127.29.84.190:38747/ using document root <none> and password file <none>
I20260806 00:48:17.143355 30308 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:17.143415 30308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:17.143572 30308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:17.145342 30308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "689c9bdab67c45d3bcb100439ceb36f2"
format_stamp: "Formatted at 2026-08-06 00:48:17 on dist-test-slave-n326"
server_key: "055a1761985346bdafe46494a3442f9f"
server_key_iv: "8e7bdf6ae7a935d41b2ba5234eca432a"
server_key_version: "encryptionkey@0"
I20260806 00:48:17.145830 30308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "689c9bdab67c45d3bcb100439ceb36f2"
format_stamp: "Formatted at 2026-08-06 00:48:17 on dist-test-slave-n326"
server_key: "055a1761985346bdafe46494a3442f9f"
server_key_iv: "8e7bdf6ae7a935d41b2ba5234eca432a"
server_key_version: "encryptionkey@0"
I20260806 00:48:17.149256 30308 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:48:17.151495 30325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:17.152634 30308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:17.152727 30308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "689c9bdab67c45d3bcb100439ceb36f2"
format_stamp: "Formatted at 2026-08-06 00:48:17 on dist-test-slave-n326"
server_key: "055a1761985346bdafe46494a3442f9f"
server_key_iv: "8e7bdf6ae7a935d41b2ba5234eca432a"
server_key_version: "encryptionkey@0"
I20260806 00:48:17.152804 30308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:17.157095 30308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:17.159639 30308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:17.159773 30308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:17.166473 30308 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:43477
I20260806 00:48:17.166501 30405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:43477 every 8 connection(s)
I20260806 00:48:17.167308 30308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:17.169811 30406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:17.172437 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30308
I20260806 00:48:17.172601 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:17.172901 30034 external_mini_cluster.cc:1428] Setting key 2f703d4bb2796c9785ce4ebe896e05b5
I20260806 00:48:17.175724 30406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2: Bootstrap starting.
I20260806 00:48:17.178304 30406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:17.178344 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:17.174260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33161 (local address 127.29.84.190:43477)
0806 00:48:17.174722 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:17.174741 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:17.174782 (+ 41us) server_negotiation.cc:407] Connection header received
0806 00:48:17.175616 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:17.175639 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:17.175977 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:17.176311 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:17.177034 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:17.177040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:17.177054 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:17.177067 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:17.177125 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:17.177151 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:17.177283 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:17.177346 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 00:48:17.177493 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":148,"threads_started":1}
I20260806 00:48:17.179250 30406 log.cc:824] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:17.181141 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:17.181344 30406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2: No bootstrap required, opened a new log
I20260806 00:48:17.184842 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:17.183222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59047 (local address 127.29.84.190:43477)
0806 00:48:17.183328 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:17.183332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:17.183574 (+ 242us) server_negotiation.cc:407] Connection header received
0806 00:48:17.183656 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:17.183659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:17.183687 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:17.183759 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:17.184454 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:17.184456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:17.184458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:17.184468 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:17.184488 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:17.184497 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:17.184565 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:17.184690 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 00:48:17.184719 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:48:17.185621 30406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "689c9bdab67c45d3bcb100439ceb36f2" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43477 } }
I20260806 00:48:17.185842 30406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:17.185899 30406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 689c9bdab67c45d3bcb100439ceb36f2, State: Initialized, Role: FOLLOWER
I20260806 00:48:17.186275 30406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "689c9bdab67c45d3bcb100439ceb36f2" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43477 } }
I20260806 00:48:17.186417 30406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:17.186486 30406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:17.186581 30406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:17.187484 30406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "689c9bdab67c45d3bcb100439ceb36f2" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43477 } }
I20260806 00:48:17.187826 30406 leader_election.cc:302] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 689c9bdab67c45d3bcb100439ceb36f2; no voters:
I20260806 00:48:17.188144 30406 leader_election.cc:288] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:17.188321 30418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:17.188592 30418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [term 1 LEADER]: Becoming Leader. State: Replica: 689c9bdab67c45d3bcb100439ceb36f2, State: Running, Role: LEADER
I20260806 00:48:17.188925 30418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "689c9bdab67c45d3bcb100439ceb36f2" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43477 } }
I20260806 00:48:17.189343 30406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:17.190834 30423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "689c9bdab67c45d3bcb100439ceb36f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "689c9bdab67c45d3bcb100439ceb36f2" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43477 } } }
I20260806 00:48:17.190968 30423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:17.191435 30424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 689c9bdab67c45d3bcb100439ceb36f2. Latest consensus state: current_term: 1 leader_uuid: "689c9bdab67c45d3bcb100439ceb36f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "689c9bdab67c45d3bcb100439ceb36f2" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43477 } } }
I20260806 00:48:17.191874 30424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2 [sys.catalog]: This master's current role is: LEADER
W20260806 00:48:17.194095 30445 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:17.194167 30445 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:48:17.194268 30440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:17.194928 30440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:17.200903 30440 catalog_manager.cc:1347] Generated new cluster ID: 0f61e217aa174bec85612c4f0233ec10
I20260806 00:48:17.200976 30440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:17.211843 30440 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:17.212122 30440 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:17.218703 30440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 689c9bdab67c45d3bcb100439ceb36f2: Generated new TSK 0
I20260806 00:48:17.219368 30440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:17.228434 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30308
2026-08-06T00:48:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:17 dist-test-slave-n326 krb5kdc[30452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:17 dist-test-slave-n326 krb5kdc[30452](info): set up 2 sockets
Aug 06 00:48:17 dist-test-slave-n326 krb5kdc[30452](info): commencing operation
krb5kdc: starting...
W20260806 00:48:18.448760 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:18 dist-test-slave-n326 krb5kdc[30452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:48:18.565399 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43675
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:40839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:43675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:18.672549 30473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:18.672822 30473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:18.672894 30473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:18.676780 30473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:18.676859 30473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:18.676898 30473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:18.676956 30473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:18.676996 30473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:18.682236 30473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:40839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:43675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:18.683285 30473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:18.684325 30473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:18.692816 30478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:18.692837 30479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:18.693066 30473 server_base.cc:1034] running on GCE node
W20260806 00:48:18.693140 30481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:18.693641 30473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:18.694665 30473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:18.695827 30473 hybrid_clock.cc:648] HybridClock initialized: now 1785977298695784 us; error 55 us; skew 500 ppm
Aug 06 00:48:18 dist-test-slave-n326 krb5kdc[30452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977298, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:18.698552 30473 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:48:18.699482 30473 webserver.cc:460] Webserver started at http://127.29.84.190:33255/ using document root <none> and password file <none>
I20260806 00:48:18.700094 30473 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:18.700160 30473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:18.700408 30473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:18.702410 30473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "5fba5e46304c449085e0e11437a3440f"
format_stamp: "Formatted at 2026-08-06 00:48:18 on dist-test-slave-n326"
server_key: "e1a2594ce531e5b652bda3b16c133c8e"
server_key_iv: "144070023c84d483a4fb3290b0911692"
server_key_version: "encryptionkey@0"
I20260806 00:48:18.702978 30473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "5fba5e46304c449085e0e11437a3440f"
format_stamp: "Formatted at 2026-08-06 00:48:18 on dist-test-slave-n326"
server_key: "e1a2594ce531e5b652bda3b16c133c8e"
server_key_iv: "144070023c84d483a4fb3290b0911692"
server_key_version: "encryptionkey@0"
I20260806 00:48:18.706457 30473 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:48:18.708858 30494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:18.710140 30473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:18.710296 30473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "5fba5e46304c449085e0e11437a3440f"
format_stamp: "Formatted at 2026-08-06 00:48:18 on dist-test-slave-n326"
server_key: "e1a2594ce531e5b652bda3b16c133c8e"
server_key_iv: "144070023c84d483a4fb3290b0911692"
server_key_version: "encryptionkey@0"
I20260806 00:48:18.710413 30473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:18.730837 30473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:18.733652 30473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:18.733842 30473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:18.740602 30473 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:43675
I20260806 00:48:18.740631 30569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:43675 every 8 connection(s)
I20260806 00:48:18.741497 30473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:18.744131 30571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:18.749219 30571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f: Bootstrap starting.
I20260806 00:48:18.750833 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30473
I20260806 00:48:18.751008 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:18.751302 30034 external_mini_cluster.cc:1428] Setting key cb887366cf1bcf9c7897899b463916a4
I20260806 00:48:18.751574 30571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:18.752499 30571 log.cc:824] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:18.754401 30571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f: No bootstrap required, opened a new log
I20260806 00:48:18.758354 30571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fba5e46304c449085e0e11437a3440f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43675 } }
I20260806 00:48:18.758651 30571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:18.758719 30571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fba5e46304c449085e0e11437a3440f, State: Initialized, Role: FOLLOWER
I20260806 00:48:18.759238 30571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fba5e46304c449085e0e11437a3440f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43675 } }
I20260806 00:48:18.759393 30571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:18.759464 30571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:18.759579 30571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [term 0 FOLLOWER]: Advancing to term 1
Aug 06 00:48:18 dist-test-slave-n326 krb5kdc[30452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:48:18.760505 30571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fba5e46304c449085e0e11437a3440f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43675 } }
I20260806 00:48:18.760836 30571 leader_election.cc:302] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fba5e46304c449085e0e11437a3440f; no voters:
I20260806 00:48:18.761189 30571 leader_election.cc:288] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:18.761296 30579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:18.761579 30579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [term 1 LEADER]: Becoming Leader. State: Replica: 5fba5e46304c449085e0e11437a3440f, State: Running, Role: LEADER
I20260806 00:48:18.761943 30579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fba5e46304c449085e0e11437a3440f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43675 } }
I20260806 00:48:18.762461 30571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:18.763680 30580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fba5e46304c449085e0e11437a3440f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fba5e46304c449085e0e11437a3440f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43675 } } }
I20260806 00:48:18.763808 30580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:18.764180 30581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fba5e46304c449085e0e11437a3440f. Latest consensus state: current_term: 1 leader_uuid: "5fba5e46304c449085e0e11437a3440f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fba5e46304c449085e0e11437a3440f" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43675 } } }
I20260806 00:48:18.764365 30581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:18.766661 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:18.752435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59381 (local address 127.29.84.190:43675)
0806 00:48:18.752889 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:18.752905 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:18.752938 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:48:18.753538 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:18.753572 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:18.753872 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:18.754134 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:18.755154 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:18.756124 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:18.757377 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:18.757806 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:18.757916 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:18.760732 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:18.760737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:18.760749 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:18.760774 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:18.763198 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:18.763767 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:18.763773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:18.763779 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:18.763918 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:18.764296 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:18.764301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:18.764303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:18.765007 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:18.765234 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:18.765495 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 00:48:18.765791 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":158,"threads_started":1}
W20260806 00:48:18.767390 30597 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:18.767448 30597 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:48:18.767527 30590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:18.768278 30590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:18.769503 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:18.773783 30590 catalog_manager.cc:1347] Generated new cluster ID: 808daebeacc74c25994f90b055564291
I20260806 00:48:18.773835 30590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:18.779975 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:18.771618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38991 (local address 127.29.84.190:43675)
0806 00:48:18.771720 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:18.771724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:18.772075 (+ 351us) server_negotiation.cc:407] Connection header received
0806 00:48:18.772201 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:18.772204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:18.772246 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:18.772306 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:18.773232 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:18.773964 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:18.774960 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:18.775325 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:18.775428 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:18.776813 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:18.776817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:18.776820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:18.776844 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:18.778350 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:18.778755 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:18.778758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:18.778760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:18.778843 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:18.779183 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:18.779186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:18.779188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:18.779357 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:18.779465 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:18.779732 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 00:48:18.779829 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 00:48:18.795320 30590 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:18.796036 30590 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:18.799638 30590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fba5e46304c449085e0e11437a3440f: Generated new TSK 0
I20260806 00:48:18.800207 30590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:18.806607 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30473
2026-08-06T00:48:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1581 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:18 dist-test-slave-n326 krb5kdc[30609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:18 dist-test-slave-n326 krb5kdc[30609](info): set up 2 sockets
Aug 06 00:48:18 dist-test-slave-n326 krb5kdc[30609](info): commencing operation
krb5kdc: starting...
W20260806 00:48:20.011149 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:20 dist-test-slave-n326 krb5kdc[30609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:48:20.133819 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:39051
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:39051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:20.244766 30624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:20.245015 30624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:20.245087 30624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:20.248939 30624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:20.249017 30624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:20.249055 30624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:20.249111 30624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:20.249147 30624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:20.254057 30624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:39051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:39051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:20.254942 30624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:20.255863 30624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:20.263546 30631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:20.263573 30633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:20.263594 30624 server_base.cc:1034] running on GCE node
W20260806 00:48:20.263584 30639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:20.264303 30624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:20.265256 30624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:20.266439 30624 hybrid_clock.cc:648] HybridClock initialized: now 1785977300266422 us; error 29 us; skew 500 ppm
Aug 06 00:48:20 dist-test-slave-n326 krb5kdc[30609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:20.268908 30624 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:48:20.269708 30624 webserver.cc:460] Webserver started at http://127.29.84.190:45895/ using document root <none> and password file <none>
I20260806 00:48:20.270231 30624 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:20.270320 30624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:20.270540 30624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:20.272394 30624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "478af8d92a694b1d9fa7cde6db2717b7"
format_stamp: "Formatted at 2026-08-06 00:48:20 on dist-test-slave-n326"
server_key: "312ae72375a068c7428cd962e7e02971"
server_key_iv: "f5d8127e53aa11fda6a6344cb118e377"
server_key_version: "encryptionkey@0"
I20260806 00:48:20.272935 30624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "478af8d92a694b1d9fa7cde6db2717b7"
format_stamp: "Formatted at 2026-08-06 00:48:20 on dist-test-slave-n326"
server_key: "312ae72375a068c7428cd962e7e02971"
server_key_iv: "f5d8127e53aa11fda6a6344cb118e377"
server_key_version: "encryptionkey@0"
I20260806 00:48:20.276278 30624 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:48:20.278411 30652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:20.279531 30624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:20.279662 30624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "478af8d92a694b1d9fa7cde6db2717b7"
format_stamp: "Formatted at 2026-08-06 00:48:20 on dist-test-slave-n326"
server_key: "312ae72375a068c7428cd962e7e02971"
server_key_iv: "f5d8127e53aa11fda6a6344cb118e377"
server_key_version: "encryptionkey@0"
I20260806 00:48:20.279767 30624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:20.305660 30624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:20.308398 30624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:20.308600 30624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:20.315575 30624 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:39051
I20260806 00:48:20.315591 30731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:39051 every 8 connection(s)
I20260806 00:48:20.316434 30624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:20.319049 30732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:20.319162 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30624
I20260806 00:48:20.319293 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:20.319603 30034 external_mini_cluster.cc:1428] Setting key 1b00cd095f8a42ed68a6f348cdca035b
I20260806 00:48:20.325526 30732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7: Bootstrap starting.
I20260806 00:48:20.328012 30732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:20.328403 30738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:20.320905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56809 (local address 127.29.84.190:39051)
0806 00:48:20.321315 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:20.321330 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:20.321363 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:48:20.322157 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:20.322176 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:20.322502 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:20.322777 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:20.323146 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:20.324076 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:20.325261 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:20.325790 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:20.325912 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:20.326692 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:20.326696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:20.326707 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:20.327156 (+ 449us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:20.327210 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:20.327232 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:20.327307 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:20.327465 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 00:48:20.327682 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":146,"threads_started":1}
I20260806 00:48:20.328761 30732 log.cc:824] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:20.330561 30732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7: No bootstrap required, opened a new log
I20260806 00:48:20.331032 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:20.334921 30732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478af8d92a694b1d9fa7cde6db2717b7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 39051 } }
I20260806 00:48:20.335191 30732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:20.335253 30732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478af8d92a694b1d9fa7cde6db2717b7, State: Initialized, Role: FOLLOWER
I20260806 00:48:20.335650 30732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478af8d92a694b1d9fa7cde6db2717b7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 39051 } }
I20260806 00:48:20.335768 30732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:20.335898 30732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:20.336023 30732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:20.336966 30738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:20.333175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58837 (local address 127.29.84.190:39051)
0806 00:48:20.333277 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:20.333281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:20.333453 (+ 172us) server_negotiation.cc:407] Connection header received
0806 00:48:20.333579 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:20.333581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:20.333613 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:20.333672 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:20.333977 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:20.334429 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:20.335489 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:20.335773 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:20.335873 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:20.336584 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:20.336587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:20.336589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:20.336600 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:20.336624 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:20.336642 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:20.336719 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:20.336804 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 00:48:20.336846 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:48:20.336930 30732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478af8d92a694b1d9fa7cde6db2717b7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 39051 } }
I20260806 00:48:20.337330 30732 leader_election.cc:302] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 478af8d92a694b1d9fa7cde6db2717b7; no voters:
I20260806 00:48:20.337663 30732 leader_election.cc:288] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:20.337728 30748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:20.337906 30748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [term 1 LEADER]: Becoming Leader. State: Replica: 478af8d92a694b1d9fa7cde6db2717b7, State: Running, Role: LEADER
I20260806 00:48:20.338215 30748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478af8d92a694b1d9fa7cde6db2717b7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 39051 } }
I20260806 00:48:20.338863 30732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:20.339334 30748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "478af8d92a694b1d9fa7cde6db2717b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478af8d92a694b1d9fa7cde6db2717b7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 39051 } } }
I20260806 00:48:20.339445 30748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:20.339679 30748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 478af8d92a694b1d9fa7cde6db2717b7. Latest consensus state: current_term: 1 leader_uuid: "478af8d92a694b1d9fa7cde6db2717b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478af8d92a694b1d9fa7cde6db2717b7" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 39051 } } }
I20260806 00:48:20.339859 30748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:20.340173 30753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:20.342723 30753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:20.347980 30753 catalog_manager.cc:1347] Generated new cluster ID: 61fbfa5546754fca92ae84eff99935a9
I20260806 00:48:20.348040 30753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:20.355592 30753 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:20.356684 30753 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:20.363727 30753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 478af8d92a694b1d9fa7cde6db2717b7: Generated new TSK 0
I20260806 00:48:20.364324 30753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:20.373265 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30624
2026-08-06T00:48:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1566 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:20 dist-test-slave-n326 krb5kdc[30779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:20 dist-test-slave-n326 krb5kdc[30779](info): set up 2 sockets
Aug 06 00:48:20 dist-test-slave-n326 krb5kdc[30779](info): commencing operation
krb5kdc: starting...
W20260806 00:48:21.582175 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:21 dist-test-slave-n326 krb5kdc[30779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:48:21.702999 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43991
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:43991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:21.809854 30799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:21.810132 30799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:21.810205 30799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:21.814082 30799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:21.814167 30799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:21.814205 30799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:21.814261 30799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:21.814317 30799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:21.819205 30799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:43991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:43991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:21.820137 30799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:21.821123 30799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:21.829228 30810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:21.829234 30808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:21.829413 30799 server_base.cc:1034] running on GCE node
W20260806 00:48:21.829234 30815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:21.830089 30799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:21.831010 30799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:21.832233 30799 hybrid_clock.cc:648] HybridClock initialized: now 1785977301832210 us; error 45 us; skew 500 ppm
Aug 06 00:48:21 dist-test-slave-n326 krb5kdc[30779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977301, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:21.834904 30799 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:48:21.835736 30799 webserver.cc:460] Webserver started at http://127.29.84.190:35473/ using document root <none> and password file <none>
I20260806 00:48:21.836228 30799 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:21.836284 30799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:21.836549 30799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:21.838392 30799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "deb6105e9fe443a392b8b2a473e244b4"
format_stamp: "Formatted at 2026-08-06 00:48:21 on dist-test-slave-n326"
server_key: "d45abce2eed668444d1d9513ebe17511"
server_key_iv: "8511c063b30d9e81272eb572adfc2898"
server_key_version: "encryptionkey@0"
I20260806 00:48:21.838943 30799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "deb6105e9fe443a392b8b2a473e244b4"
format_stamp: "Formatted at 2026-08-06 00:48:21 on dist-test-slave-n326"
server_key: "d45abce2eed668444d1d9513ebe17511"
server_key_iv: "8511c063b30d9e81272eb572adfc2898"
server_key_version: "encryptionkey@0"
I20260806 00:48:21.842326 30799 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260806 00:48:21.844558 30826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:21.845671 30799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:48:21.845805 30799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "deb6105e9fe443a392b8b2a473e244b4"
format_stamp: "Formatted at 2026-08-06 00:48:21 on dist-test-slave-n326"
server_key: "d45abce2eed668444d1d9513ebe17511"
server_key_iv: "8511c063b30d9e81272eb572adfc2898"
server_key_version: "encryptionkey@0"
I20260806 00:48:21.845907 30799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:21.850075 30799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:21.852654 30799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:21.852847 30799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:21.859259 30799 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:43991
I20260806 00:48:21.859323 30907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:43991 every 8 connection(s)
I20260806 00:48:21.860116 30799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:21.862636 30909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:21.867429 30909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4: Bootstrap starting.
I20260806 00:48:21.868150 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30799
I20260806 00:48:21.868288 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:21.868543 30034 external_mini_cluster.cc:1428] Setting key fe7096c8c4fc426e6737bf39c1cb5f3b
I20260806 00:48:21.869993 30909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:21.870849 30909 log.cc:824] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:21.872691 30909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4: No bootstrap required, opened a new log
I20260806 00:48:21.873488 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:21.869531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37435 (local address 127.29.84.190:43991)
0806 00:48:21.870018 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:21.870039 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:21.870081 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:48:21.870871 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:21.870892 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:21.871268 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:21.871578 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:21.872331 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:21.872336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:21.872356 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:21.872366 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:21.872411 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:21.872428 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:21.872535 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:21.872687 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 00:48:21.872849 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260806 00:48:21.876004 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:21.876305 30909 raft_consensus.cc:348] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb6105e9fe443a392b8b2a473e244b4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43991 } }
I20260806 00:48:21.876521 30909 raft_consensus.cc:374] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:21.876600 30909 raft_consensus.cc:729] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deb6105e9fe443a392b8b2a473e244b4, State: Initialized, Role: FOLLOWER
I20260806 00:48:21.876979 30909 consensus_queue.cc:260] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb6105e9fe443a392b8b2a473e244b4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43991 } }
I20260806 00:48:21.877146 30909 raft_consensus.cc:388] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:21.877230 30909 raft_consensus.cc:482] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:21.877344 30909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:21.878175 30909 raft_consensus.cc:504] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb6105e9fe443a392b8b2a473e244b4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43991 } }
I20260806 00:48:21.878515 30909 leader_election.cc:302] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: deb6105e9fe443a392b8b2a473e244b4; no voters:
I20260806 00:48:21.878856 30909 leader_election.cc:288] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:21.878947 30928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:21.879171 30928 raft_consensus.cc:686] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [term 1 LEADER]: Becoming Leader. State: Replica: deb6105e9fe443a392b8b2a473e244b4, State: Running, Role: LEADER
I20260806 00:48:21.879463 30928 consensus_queue.cc:237] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb6105e9fe443a392b8b2a473e244b4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43991 } }
I20260806 00:48:21.879658 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:21.878081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57689 (local address 127.29.84.190:43991)
0806 00:48:21.878181 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:21.878184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:21.878321 (+ 137us) server_negotiation.cc:407] Connection header received
0806 00:48:21.878466 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:21.878469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:21.878495 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:21.878554 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:21.879280 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:21.879283 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:21.879285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:21.879295 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:21.879317 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:21.879328 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:21.879375 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:21.879515 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 00:48:21.879565 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:48:21.880050 30909 sys_catalog.cc:564] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:21.881078 30928 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader deb6105e9fe443a392b8b2a473e244b4. Latest consensus state: current_term: 1 leader_uuid: "deb6105e9fe443a392b8b2a473e244b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb6105e9fe443a392b8b2a473e244b4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43991 } } }
I20260806 00:48:21.881254 30928 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:21.881506 30930 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "deb6105e9fe443a392b8b2a473e244b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb6105e9fe443a392b8b2a473e244b4" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 43991 } } }
I20260806 00:48:21.881613 30930 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:21.881732 30942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:21.884325 30942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:21.889213 30942 catalog_manager.cc:1347] Generated new cluster ID: d7e52a502cc64fcf9034c8ef15b43d0d
I20260806 00:48:21.889272 30942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:21.897766 30942 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:21.898030 30942 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:21.912040 30942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P deb6105e9fe443a392b8b2a473e244b4: Generated new TSK 0
I20260806 00:48:21.912655 30942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:21.924328 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30799
2026-08-06T00:48:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1545 ms)
[----------] 18 tests from AuthTokenIssuingTest (28145 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:21 dist-test-slave-n326 krb5kdc[30956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:21 dist-test-slave-n326 krb5kdc[30956](info): set up 2 sockets
Aug 06 00:48:21 dist-test-slave-n326 krb5kdc[30956](info): commencing operation
krb5kdc: starting...
W20260806 00:48:23.134459 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:48:23.253399 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:44819
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:44819,127.29.84.189:43347,127.29.84.188:40699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:23.360570 30975 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:48:23.360946 30975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:23.360997 30975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:23.361068 30975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:23.365198 30975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:23.365284 30975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:23.365320 30975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:23.365360 30975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:23.365394 30975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:23.370569 30975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:44819,127.29.84.189:43347,127.29.84.188:40699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:44819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:23.371649 30975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:23.372627 30975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:23.380779 30988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:23.380779 30990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:23.380772 30984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:23.381147 30975 server_base.cc:1034] running on GCE node
I20260806 00:48:23.381661 30975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:23.382699 30975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:23.383855 30975 hybrid_clock.cc:648] HybridClock initialized: now 1785977303383836 us; error 31 us; skew 500 ppm
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:23.386547 30975 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:48:23.387391 30975 webserver.cc:460] Webserver started at http://127.29.84.190:32845/ using document root <none> and password file <none>
I20260806 00:48:23.387985 30975 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:23.388067 30975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:23.388273 30975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:23.390311 30975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "e9b7b3873d05425eb2af0d3da76ad444"
format_stamp: "Formatted at 2026-08-06 00:48:23 on dist-test-slave-n326"
server_key: "e5337ea34ba9d55062bcf047838ebf9e"
server_key_iv: "05cb0a05320b9c376ffbd59d63b23a20"
server_key_version: "encryptionkey@0"
I20260806 00:48:23.390882 30975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "e9b7b3873d05425eb2af0d3da76ad444"
format_stamp: "Formatted at 2026-08-06 00:48:23 on dist-test-slave-n326"
server_key: "e5337ea34ba9d55062bcf047838ebf9e"
server_key_iv: "05cb0a05320b9c376ffbd59d63b23a20"
server_key_version: "encryptionkey@0"
I20260806 00:48:23.394481 30975 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:48:23.396809 30998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:23.397945 30975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 00:48:23.398093 30975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "e9b7b3873d05425eb2af0d3da76ad444"
format_stamp: "Formatted at 2026-08-06 00:48:23 on dist-test-slave-n326"
server_key: "e5337ea34ba9d55062bcf047838ebf9e"
server_key_iv: "05cb0a05320b9c376ffbd59d63b23a20"
server_key_version: "encryptionkey@0"
I20260806 00:48:23.398213 30975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:23.417888 30975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:23.421208 30975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:23.421424 30975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:23.428181 30975 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:44819
I20260806 00:48:23.428459 31074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:44819 every 8 connection(s)
I20260806 00:48:23.429147 30975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:23.431825 31077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:23.435349 31077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:48:23.438890 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 30975
I20260806 00:48:23.439016 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:23.439347 30034 external_mini_cluster.cc:1428] Setting key cf1954896183ff7a4896da6da9a495b4
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.84.189@KRBTEST.COM; defaulting to no policy
I20260806 00:48:23.451426 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.437503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.29.84.190:44819)
0806 00:48:23.437999 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.438028 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.438190 (+ 162us) server_negotiation.cc:407] Connection header received
0806 00:48:23.439418 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.439422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.439634 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.439777 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.440915 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.442200 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.443207 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.443802 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.443999 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.446577 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.446582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.446591 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.446611 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.448233 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.448822 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.448826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.448828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.448923 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.449303 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.449306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.449307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.449624 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.449722 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.450185 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.450407 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"spinlock_wait_cycles":32128,"thread_start_us":154,"threads_started":1}
I20260806 00:48:23.451426 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.437503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44819 (local address 127.0.0.1:42976)
0806 00:48:23.438026 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.438073 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:23.439109 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:23.439847 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:23.439855 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:23.440364 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:23.440751 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.440772 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.442336 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.442339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.443051 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.443059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.443945 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.443948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.444035 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.444612 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:23.444635 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:23.446431 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:23.448373 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.448386 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.448398 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.448714 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.449043 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.449048 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.449052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.449197 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.449724 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:23.449729 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:23.449841 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 00:48:23.450109 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 00:48:23.450484 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":104,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.189@KRBTEST.COM" created.
I20260806 00:48:23.453998 31077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:48:23.454761 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.454443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43347 (local address 127.0.0.1:34698)
0806 00:48:23.454580 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.454675 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 00:48:23.454994 31002 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
I20260806 00:48:23.455930 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.455647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43347 (local address 127.0.0.1:34706)
0806 00:48:23.455788 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.455849 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 00:48:23.456130 31077 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.189:43347: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab.
Entry for principal kudu/127.29.84.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab.
I20260806 00:48:23.469767 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.189:43347
--webserver_interface=127.29.84.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab
--principal=kudu/127.29.84.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:44819,127.29.84.189:43347,127.29.84.188:40699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:48:23.505354 31077 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } attempt: 1
I20260806 00:48:23.506001 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.505782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43347 (local address 127.0.0.1:34718)
0806 00:48:23.505898 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.505945 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 00:48:23.506718 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.506540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43347 (local address 127.0.0.1:34732)
0806 00:48:23.506629 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.506669 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 00:48:23.506901 31077 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.189:43347: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
W20260806 00:48:23.573764 31085 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:48:23.574033 31085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:23.574116 31085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:23.574193 31085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 00:48:23.575042 31077 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } attempt: 2
I20260806 00:48:23.575802 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.575549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43347 (local address 127.0.0.1:34734)
0806 00:48:23.575672 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.575724 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 00:48:23.576483 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.576283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43347 (local address 127.0.0.1:34740)
0806 00:48:23.576385 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.576435 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 00:48:23.576619 31077 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.189:43347: Network error: Client connection negotiation failed: client connection to 127.29.84.189:43347: connect: Connection refused (error 111)
W20260806 00:48:23.578481 31085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:23.578523 31085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:23.578543 31085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:23.578594 31085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:23.578624 31085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:23.583945 31085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:44819,127.29.84.189:43347,127.29.84.188:40699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab
--principal=kudu/127.29.84.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.189:43347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.84.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:23.584777 31085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:23.585618 31085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:23.593456 31094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:23.593456 31097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:23.593510 31085 server_base.cc:1034] running on GCE node
W20260806 00:48:23.593504 31095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:23.594219 31085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:23.595189 31085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:23.596360 31085 hybrid_clock.cc:648] HybridClock initialized: now 1785977303596337 us; error 36 us; skew 500 ppm
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:23.599010 31085 init.cc:377] Logged in from keytab as kudu/127.29.84.189@KRBTEST.COM (short username kudu)
I20260806 00:48:23.599823 31085 webserver.cc:460] Webserver started at http://127.29.84.189:40873/ using document root <none> and password file <none>
I20260806 00:48:23.600332 31085 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:23.600390 31085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:23.600625 31085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:23.602620 31085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/data/instance:
uuid: "a2114a3b3ece47bd9b3d5a6b257df297"
format_stamp: "Formatted at 2026-08-06 00:48:23 on dist-test-slave-n326"
server_key: "93f1220288199e64ddee3741aebcac9b"
server_key_iv: "348f2da622adb7eed80c9cf0d2f90a83"
server_key_version: "encryptionkey@0"
I20260806 00:48:23.603145 31085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/wal/instance:
uuid: "a2114a3b3ece47bd9b3d5a6b257df297"
format_stamp: "Formatted at 2026-08-06 00:48:23 on dist-test-slave-n326"
server_key: "93f1220288199e64ddee3741aebcac9b"
server_key_iv: "348f2da622adb7eed80c9cf0d2f90a83"
server_key_version: "encryptionkey@0"
I20260806 00:48:23.606381 31085 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:48:23.608528 31108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:23.609632 31085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:48:23.609722 31085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/wal
uuid: "a2114a3b3ece47bd9b3d5a6b257df297"
format_stamp: "Formatted at 2026-08-06 00:48:23 on dist-test-slave-n326"
server_key: "93f1220288199e64ddee3741aebcac9b"
server_key_iv: "348f2da622adb7eed80c9cf0d2f90a83"
server_key_version: "encryptionkey@0"
I20260806 00:48:23.609794 31085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:23.622503 31085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:23.625363 31085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:23.625500 31085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:23.632278 31085 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.189:43347
I20260806 00:48:23.632542 31184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.189:43347 every 8 connection(s)
I20260806 00:48:23.633127 31085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
I20260806 00:48:23.635327 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 31085
I20260806 00:48:23.635452 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-1/wal/instance
I20260806 00:48:23.635696 31185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:23.635787 30034 external_mini_cluster.cc:1428] Setting key b9db0828a233b44ef7c41d6b849686b1
I20260806 00:48:23.639575 31185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.84.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.188@KRBTEST.COM" created.
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:48:23.652681 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.641328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.29.84.190:44819)
0806 00:48:23.641518 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.641526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.642220 (+ 694us) server_negotiation.cc:407] Connection header received
0806 00:48:23.643103 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.643109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.643169 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.643261 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.644359 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.644884 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.646059 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.646328 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.646436 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.648991 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.648994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.648996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.649019 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.650420 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.651115 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.651118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.651119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.651199 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.651604 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.651606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.651607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.651766 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.651839 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.652469 (+ 630us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.652571 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:48:23.653847 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.641632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44819 (local address 127.0.0.1:42980)
0806 00:48:23.642054 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.642104 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:23.642869 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:23.643409 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:23.643415 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:23.643722 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:23.644212 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.644227 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.644986 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.644989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.645951 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.645961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.646491 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.646495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.646651 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.647172 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:23.647200 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:23.648819 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:23.650562 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.650570 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.650586 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.650949 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.651317 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.651323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.651327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.651504 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.651881 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:23.651888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:23.652201 (+ 313us) client_negotiation.cc:769] Sending connection context
0806 00:48:23.652417 (+ 216us) client_negotiation.cc:240] Negotiation successful
0806 00:48:23.652732 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":115,"threads_started":1}
I20260806 00:48:23.655205 31185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:48:23.664117 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.655661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43347 (local address 127.0.0.1:34744)
0806 00:48:23.655821 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.655837 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:23.655909 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:23.656376 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:23.656378 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:23.656494 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:23.656574 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.656578 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.657613 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.657615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.658343 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.658355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.658918 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.658921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.659000 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.659404 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:23.659415 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:23.660723 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:23.662474 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.662477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.662478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.662774 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.663103 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.663108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.663110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.663188 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.663639 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:23.663642 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:23.663725 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 00:48:23.663838 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 00:48:23.663954 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 00:48:23.664172 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.655772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34744 (local address 127.29.84.189:43347)
0806 00:48:23.656051 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.656061 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.656081 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:48:23.656118 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.656121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.656243 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.656338 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.656670 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.657484 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.658466 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.658811 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.658915 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.660846 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.660854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.660862 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.660883 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.662379 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.662917 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.662921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.662922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.662995 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.663312 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.663317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.663318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.663530 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.663661 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.663925 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.664034 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":135,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab.
Entry for principal kudu/127.29.84.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab.
I20260806 00:48:23.665778 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.188:40699
--webserver_interface=127.29.84.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab
--principal=kudu/127.29.84.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:44819,127.29.84.189:43347,127.29.84.188:40699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:48:23.665953 31185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:48:23.666570 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.666361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56894)
0806 00:48:23.666466 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.666524 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 00:48:23.666818 31115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
I20260806 00:48:23.667678 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.667468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56896)
0806 00:48:23.667583 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.667627 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 00:48:23.667793 31185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:40699: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
I20260806 00:48:23.667798 31077 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } attempt: 3
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:48:23.675963 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.668237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.29.84.189:43347)
0806 00:48:23.668358 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.668364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.668379 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:48:23.668449 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.668452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.668498 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.668558 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.669056 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.669505 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.670706 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.670920 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.670999 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.673041 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.673049 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.673051 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.673070 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.674404 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.674840 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.674843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.674845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.674915 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.675227 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.675230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.675232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.675390 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.675502 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.675741 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.675838 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:48:23.675966 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.668139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43347 (local address 127.0.0.1:34756)
0806 00:48:23.668309 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.668321 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:23.668389 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:23.668600 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:23.668606 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:23.668793 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:23.668942 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.668953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.669617 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.669623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.670584 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.670598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.671039 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.671046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.671205 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.671619 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:23.671639 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:23.672905 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:23.674539 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.674542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.674544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.674736 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.675007 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.675010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.675012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.675144 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.675489 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:23.675492 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:23.675588 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 00:48:23.675693 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 00:48:23.675819 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 00:48:23.676966 31077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:48:23.677613 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.677375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56912)
0806 00:48:23.677486 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.677548 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 00:48:23.678244 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.678000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56914)
0806 00:48:23.678140 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.678187 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260806 00:48:23.678372 31077 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:40699: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
I20260806 00:48:23.709532 31077 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } attempt: 1
I20260806 00:48:23.710167 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.709908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56930)
0806 00:48:23.710036 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.710100 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 00:48:23.710876 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.710653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56932)
0806 00:48:23.710749 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.710811 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:48:23.711018 31077 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:40699: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
I20260806 00:48:23.717063 31185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } attempt: 1
I20260806 00:48:23.717617 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.717411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56938)
0806 00:48:23.717525 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.717568 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 00:48:23.718236 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.718050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56952)
0806 00:48:23.718145 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.718187 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 00:48:23.718351 31185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:40699: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
W20260806 00:48:23.773267 31194 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:48:23.773583 31194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:23.773659 31194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:23.773742 31194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:23.777551 31194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:23.777621 31194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:23.777654 31194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:23.777688 31194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:23.777742 31194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:23.782827 31194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:44819,127.29.84.189:43347,127.29.84.188:40699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab
--principal=kudu/127.29.84.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.188:40699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.84.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:23.783766 31194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:23.784637 31194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:48:23.786194 31077 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } attempt: 2
I20260806 00:48:23.786486 31185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } attempt: 2
I20260806 00:48:23.787009 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.786659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56968)
0806 00:48:23.786846 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.786908 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 00:48:23.787207 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.786892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56978)
0806 00:48:23.787047 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.787135 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 00:48:23.787955 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.787704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:56990)
0806 00:48:23.787822 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.787884 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 00:48:23.788110 31185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:40699: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
I20260806 00:48:23.788566 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.788320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:57002)
0806 00:48:23.788450 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.788501 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 00:48:23.788725 31077 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:40699: Network error: Client connection negotiation failed: client connection to 127.29.84.188:40699: connect: Connection refused (error 111)
W20260806 00:48:23.793454 31208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:23.793426 31209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:23.793573 31194 server_base.cc:1034] running on GCE node
W20260806 00:48:23.793473 31213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:23.794299 31194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:23.795245 31194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:23.796417 31194 hybrid_clock.cc:648] HybridClock initialized: now 1785977303796399 us; error 27 us; skew 500 ppm
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:23.798952 31194 init.cc:377] Logged in from keytab as kudu/127.29.84.188@KRBTEST.COM (short username kudu)
I20260806 00:48:23.799806 31194 webserver.cc:460] Webserver started at http://127.29.84.188:42437/ using document root <none> and password file <none>
I20260806 00:48:23.800375 31194 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:23.800441 31194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:23.800693 31194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:23.802810 31194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/data/instance:
uuid: "afb335cd15fa43b69b377e832b009a98"
format_stamp: "Formatted at 2026-08-06 00:48:23 on dist-test-slave-n326"
server_key: "5d3bf2be78e7b38c8c30881b27c4e4be"
server_key_iv: "cbaeab44cf923eb4ac51589e329b9742"
server_key_version: "encryptionkey@0"
I20260806 00:48:23.803406 31194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/wal/instance:
uuid: "afb335cd15fa43b69b377e832b009a98"
format_stamp: "Formatted at 2026-08-06 00:48:23 on dist-test-slave-n326"
server_key: "5d3bf2be78e7b38c8c30881b27c4e4be"
server_key_iv: "cbaeab44cf923eb4ac51589e329b9742"
server_key_version: "encryptionkey@0"
I20260806 00:48:23.806777 31194 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:48:23.808984 31221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:23.810159 31194 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:23.810290 31194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/wal
uuid: "afb335cd15fa43b69b377e832b009a98"
format_stamp: "Formatted at 2026-08-06 00:48:23 on dist-test-slave-n326"
server_key: "5d3bf2be78e7b38c8c30881b27c4e4be"
server_key_iv: "cbaeab44cf923eb4ac51589e329b9742"
server_key_version: "encryptionkey@0"
I20260806 00:48:23.810392 31194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:23.841958 31194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:23.844604 31194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:23.844800 31194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:23.851969 31194 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.188:40699
I20260806 00:48:23.852221 31289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.188:40699 every 8 connection(s)
I20260806 00:48:23.852885 31194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
I20260806 00:48:23.855517 31292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:23.859040 31292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:48:23.862195 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 31194
I20260806 00:48:23.862340 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977020829091-30034-0/minicluster-data/master-2/wal/instance
I20260806 00:48:23.862602 30034 external_mini_cluster.cc:1428] Setting key 7711d89452cd99a6a61aa2310deece94
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
I20260806 00:48:23.872390 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.860827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42988 (local address 127.29.84.190:44819)
0806 00:48:23.860959 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.860962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.861764 (+ 802us) server_negotiation.cc:407] Connection header received
0806 00:48:23.862697 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.862703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.862761 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.862863 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.864195 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.864696 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.865737 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.865967 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.866075 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.868673 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.868676 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.868678 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.868701 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.870278 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.870790 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.870794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.870796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.870880 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.871292 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.871296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.871298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.871485 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.871586 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.872114 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.872230 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:48:23.873131 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.863297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.29.84.188:40699)
0806 00:48:23.863695 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.863707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.863730 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:48:23.863841 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.863847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.864001 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.864159 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.864870 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.865645 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.866471 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.866810 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.866920 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.869342 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.869346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.869355 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.869374 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.870871 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.871386 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.871403 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.871405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.871477 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.871765 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.871768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.871770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.871998 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.872218 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.872385 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.872722 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":136,"threads_started":1}
I20260806 00:48:23.873138 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.861163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:44819 (local address 127.0.0.1:42988)
0806 00:48:23.861616 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.861657 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:23.862485 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:23.863066 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:23.863094 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:23.863609 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:23.864045 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.864057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.864827 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.864835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.865610 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.865619 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.866118 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.866122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.866283 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.866815 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:23.866831 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:23.868519 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:23.870396 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.870408 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.870416 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.870681 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.870984 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.870988 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.870990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.871192 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.871600 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:23.871607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:23.871865 (+ 258us) client_negotiation.cc:769] Sending connection context
0806 00:48:23.872035 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 00:48:23.872310 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"spinlock_wait_cycles":8960,"thread_start_us":109,"threads_started":1}
I20260806 00:48:23.874294 31292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:48:23.879279 31185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } attempt: 3
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:48:23.883324 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.874724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:43347 (local address 127.0.0.1:34764)
0806 00:48:23.874824 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.874838 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:23.874907 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:23.875209 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:23.875212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:23.875356 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:23.875431 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.875436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.876447 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.876450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.876987 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.876996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.877431 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.877435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.877531 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.877962 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:23.877976 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:23.879380 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:23.881610 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.881613 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.881614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.881901 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.882229 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.882232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.882234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.882324 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.882807 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:23.882810 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:23.882920 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 00:48:23.883037 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 00:48:23.883183 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 00:48:23.883652 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.874851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.29.84.189:43347)
0806 00:48:23.874975 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.874979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.874991 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:48:23.875027 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.875029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.875117 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.875193 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.875581 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.876324 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.877101 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.877321 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.877439 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.879541 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.879544 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.879546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.879575 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.881483 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.882016 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.882023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.882027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.882120 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.882441 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.882444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.882446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.882665 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.882732 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.883144 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.883248 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:48:23.884160 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.875586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.29.84.190:44819)
0806 00:48:23.875687 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.875690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.875706 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:48:23.875804 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.875806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.875835 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.875891 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.876544 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.876985 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.877702 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.877916 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.878016 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.880577 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.880581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.880583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.880605 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.882284 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.882713 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.882716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.882717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.882805 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.883144 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.883147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.883149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.883352 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.883542 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.883910 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.884021 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:48:23.884405 31292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
I20260806 00:48:23.887894 31077 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } attempt: 3
I20260806 00:48:23.888521 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.879727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:57014)
0806 00:48:23.879891 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.879904 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:23.879969 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:23.880458 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:23.880462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:23.880632 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:23.880716 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.880721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.881740 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.881743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.882510 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.882519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.883135 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.883139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.883225 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.883642 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:23.883656 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:23.885141 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:23.887041 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.887044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.887046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.887301 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.887583 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.887586 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.887588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.887683 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.888078 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:23.888081 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:23.888162 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 00:48:23.888235 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:48:23.888349 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 00:48:23.889096 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.880079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.29.84.188:40699)
0806 00:48:23.880250 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.880256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.880271 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:48:23.880309 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.880312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.880351 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.880434 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.880838 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.881614 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.882636 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.882956 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.883419 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.885301 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.885304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.885307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.885330 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.886938 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.887392 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.887395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.887397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.887475 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.887775 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.887778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.887780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.887956 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.888767 (+ 811us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.888843 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.888936 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:48:23.892805 31185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297: Bootstrap starting.
Aug 06 00:48:23 dist-test-slave-n326 krb5kdc[30956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
I20260806 00:48:23.894215 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.884808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:57024)
0806 00:48:23.885187 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.885200 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:23.885293 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:23.885528 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:23.885534 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:23.885681 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:23.885772 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.885780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.887263 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.887269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.887951 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.887957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.888594 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.888597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.888681 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.889074 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:23.889087 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:23.890424 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:23.892724 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.892727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.892728 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.892957 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.893237 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.893239 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.893241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.893319 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.893754 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:23.893756 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:23.893847 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 00:48:23.893937 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 00:48:23.894078 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305}
I20260806 00:48:23.894757 31299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.884965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57024 (local address 127.29.84.188:40699)
0806 00:48:23.885203 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.885212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.885258 (+ 46us) server_negotiation.cc:407] Connection header received
0806 00:48:23.885358 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.885362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.885394 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.885496 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.885897 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.887171 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.888076 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.888408 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.888495 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.890541 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.890544 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.890548 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.890568 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.892635 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.893070 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.893072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.893074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.893149 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.893411 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.893414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.893415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.893621 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.893718 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.894473 (+ 755us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.894571 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":102,"threads_started":1}
I20260806 00:48:23.895759 31185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:23.896692 31185 log.cc:824] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:23.897117 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.885695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.29.84.189:43347)
0806 00:48:23.885810 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.885813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.886059 (+ 246us) server_negotiation.cc:407] Connection header received
0806 00:48:23.886191 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.886197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.886244 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.886293 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.886932 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.887604 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.888610 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.889262 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.889344 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.892482 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.892486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.892488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.892522 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.894340 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.894821 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.894825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.894826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.895174 (+ 348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.895748 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.895751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.895752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.895999 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.896174 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.896836 (+ 662us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.896937 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:48:23.897505 31292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98: Bootstrap starting.
I20260806 00:48:23.898138 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.888236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:40699 (local address 127.0.0.1:57028)
0806 00:48:23.888445 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:23.888458 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:23.888532 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:23.889867 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:23.889872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:23.890031 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:23.890157 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.890170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.891227 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.891231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.892084 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:23.892099 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.892740 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.892743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:23.892827 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.893212 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:23.893229 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:23.894551 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:23.896492 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.896495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.896497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.896759 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.897097 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:23.897101 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:23.897102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:23.897189 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:23.897666 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:23.897669 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:23.897759 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 00:48:23.897867 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 00:48:23.897987 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260806 00:48:23.898150 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.888858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57028 (local address 127.29.84.188:40699)
0806 00:48:23.889648 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.889654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.889666 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:48:23.889694 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.889697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.889732 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.890275 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.890358 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.891070 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.892243 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.892612 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.893043 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.894684 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.894687 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.894689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.894709 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.896367 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.896887 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.896890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.896892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.896971 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.897302 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.897304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.897306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.897532 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.897662 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.897930 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.898024 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":83,"threads_started":1}
I20260806 00:48:23.899375 31185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297: No bootstrap required, opened a new log
I20260806 00:48:23.899883 31292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:23.900695 31292 log.cc:824] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:23.902014 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:23.902184 31077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444: Bootstrap starting.
I20260806 00:48:23.903582 31185 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } }
I20260806 00:48:23.903793 31185 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2114a3b3ece47bd9b3d5a6b257df297, State: Initialized, Role: FOLLOWER
I20260806 00:48:23.904107 31185 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } }
I20260806 00:48:23.904309 31077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:23.903558 31038 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e9b7b3873d05425eb2af0d3da76ad444"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42996
I20260806 00:48:23.905051 31077 log.cc:824] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:23.905469 31185 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:23.907038 31077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444: No bootstrap required, opened a new log
I20260806 00:48:23.907147 31292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98: No bootstrap required, opened a new log
I20260806 00:48:23.909977 31077 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } }
I20260806 00:48:23.910200 31077 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9b7b3873d05425eb2af0d3da76ad444, State: Initialized, Role: FOLLOWER
I20260806 00:48:23.910506 31077 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } }
I20260806 00:48:23.911056 31292 raft_consensus.cc:348] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } }
I20260806 00:48:23.911345 31292 raft_consensus.cc:729] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afb335cd15fa43b69b377e832b009a98, State: Initialized, Role: FOLLOWER
I20260806 00:48:23.911646 31292 consensus_queue.cc:260] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } }
I20260806 00:48:23.911789 31077 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:23.911857 31038 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 00:48:23.911958 31038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:23.912752 31292 sys_catalog.cc:564] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:23.908977 31085 master_runner.cc:186] Error getting master registration for 127.29.84.190:44819: OK, instance_id { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" instance_seqno: 1785977303426249 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:48:23.913064 31085 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:48:23.913146 31038 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } }
I20260806 00:48:23.913460 31307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.913610 31307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:23.914088 31038 leader_election.cc:288] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [CANDIDATE]: Term 1 election: Requested vote from peers a2114a3b3ece47bd9b3d5a6b257df297 (127.29.84.189:43347), afb335cd15fa43b69b377e832b009a98 (127.29.84.188:40699)
I20260806 00:48:23.915473 31329 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.915759 31329 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:23.914702 31254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9b7b3873d05425eb2af0d3da76ad444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "afb335cd15fa43b69b377e832b009a98"
I20260806 00:48:23.916312 31254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:23.916704 31153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9b7b3873d05425eb2af0d3da76ad444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a2114a3b3ece47bd9b3d5a6b257df297"
I20260806 00:48:23.916932 31153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:23.918105 31254 raft_consensus.cc:2454] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9b7b3873d05425eb2af0d3da76ad444 in term 1.
I20260806 00:48:23.918251 31153 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9b7b3873d05425eb2af0d3da76ad444 in term 1.
I20260806 00:48:23.918493 31005 leader_election.cc:302] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afb335cd15fa43b69b377e832b009a98, e9b7b3873d05425eb2af0d3da76ad444; no voters:
W20260806 00:48:23.918715 31359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:23.918834 31359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:48:23.919185 31367 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.919339 31367 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:23.919648 31367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:23.919751 31367 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [term 1 LEADER]: Becoming Leader. State: Replica: e9b7b3873d05425eb2af0d3da76ad444, State: Running, Role: LEADER
W20260806 00:48:23.919781 31324 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:23.919924 31324 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:48:23.920045 31367 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } }
W20260806 00:48:23.923197 31007 tablet.cc:2369] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:48:23.923237 31194 master_runner.cc:263] No leader master found from master afb335cd15fa43b69b377e832b009a98
I20260806 00:48:23.923290 31194 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:48:23.923957 31326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9b7b3873d05425eb2af0d3da76ad444. Latest consensus state: current_term: 1 leader_uuid: "e9b7b3873d05425eb2af0d3da76ad444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.924141 31326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:23.924348 30975 master_runner.cc:263] No leader master found from master e9b7b3873d05425eb2af0d3da76ad444
I20260806 00:48:23.924409 30975 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 00:48:23.926921 31357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:23.927098 31357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:48:23.931387 31373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:23.934798 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.919454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34790 (local address 127.29.84.189:43347)
0806 00:48:23.919572 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.919576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.923986 (+ 4410us) server_negotiation.cc:407] Connection header received
0806 00:48:23.924152 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.924156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.924196 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.924245 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.925209 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.925946 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.927187 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.927502 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.927589 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.929354 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.929358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.929360 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.929383 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.931337 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.931841 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.931845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.931847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.932196 (+ 349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.932557 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.932561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.932563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.932768 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.933154 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.934519 (+ 1365us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.934640 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:48:23.935640 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.920770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.29.84.188:40699)
0806 00:48:23.920896 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.920900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.923865 (+ 2965us) server_negotiation.cc:407] Connection header received
0806 00:48:23.923925 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.923929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.923973 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.924046 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.925338 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.926031 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.928840 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.929164 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.929236 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.931689 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.931693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.931695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.931719 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.933737 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.934226 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.934230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.934232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.934317 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.934642 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.934646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.934648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.934851 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.934955 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.935321 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.935437 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:48:23.939468 31373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:23.940991 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:23.919619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43010 (local address 127.29.84.190:44819)
0806 00:48:23.919710 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:23.919714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:23.923582 (+ 3868us) server_negotiation.cc:407] Connection header received
0806 00:48:23.923628 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:23.923632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:23.923679 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:23.923765 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:23.927860 (+ 4095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.928559 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.932066 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:23.932392 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:23.932470 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:23.937274 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:23.937278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:23.937281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:23.937305 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:23.938954 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.939498 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.939505 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.939508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.939603 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:23.939954 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:23.939958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:23.939960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:23.940219 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:23.940349 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:23.940719 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 00:48:23.940833 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 00:48:23.946252 31254 raft_consensus.cc:1261] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [term 1 FOLLOWER]: Refusing update from remote peer e9b7b3873d05425eb2af0d3da76ad444: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:23.947368 31151 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [term 1 FOLLOWER]: Refusing update from remote peer e9b7b3873d05425eb2af0d3da76ad444: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:23.948292 31367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:23.948720 31367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:23.952113 31307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9b7b3873d05425eb2af0d3da76ad444. Latest consensus state: current_term: 1 leader_uuid: "e9b7b3873d05425eb2af0d3da76ad444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.952239 31307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:23.952521 31329 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9b7b3873d05425eb2af0d3da76ad444. Latest consensus state: current_term: 1 leader_uuid: "e9b7b3873d05425eb2af0d3da76ad444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.952651 31329 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:23.957358 31326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9b7b3873d05425eb2af0d3da76ad444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.957486 31326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:23.958531 31307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9b7b3873d05425eb2af0d3da76ad444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.958657 31307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:23.959028 31376 mvcc.cc:205] Tried to move back new op lower bound from 7315363036957544448 to 7315363036864184320. Current Snapshot: MvccSnapshot[applied={T|T < 7315363036957544448}]
I20260806 00:48:23.959503 31377 mvcc.cc:205] Tried to move back new op lower bound from 7315363036957544448 to 7315363036864184320. Current Snapshot: MvccSnapshot[applied={T|T < 7315363036957544448}]
I20260806 00:48:23.961826 31367 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9b7b3873d05425eb2af0d3da76ad444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.961926 31367 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:23.962872 31329 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9b7b3873d05425eb2af0d3da76ad444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b7b3873d05425eb2af0d3da76ad444" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 44819 } } peers { permanent_uuid: "a2114a3b3ece47bd9b3d5a6b257df297" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 43347 } } peers { permanent_uuid: "afb335cd15fa43b69b377e832b009a98" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 40699 } } }
I20260806 00:48:23.962980 31329 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:23.971912 31373 catalog_manager.cc:1347] Generated new cluster ID: 1a3fe055ce43469f86ea9ca2377013fe
I20260806 00:48:23.971972 31373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:24.025998 31373 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:24.027405 31373 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:24.041436 31373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9b7b3873d05425eb2af0d3da76ad444: Generated new TSK 0
I20260806 00:48:24.042248 31373 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:48:24.068300 31081 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.066018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.29.84.190:44819)
0806 00:48:24.066118 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.066122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.067433 (+ 1311us) server_negotiation.cc:407] Connection header received
0806 00:48:24.067592 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.067595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.067635 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.067680 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.068191 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43026: BlockingRecv error: recv got EOF from 127.0.0.1:43026 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 00:48:24.069687 31298 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.066602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57052 (local address 127.29.84.188:40699)
0806 00:48:24.066714 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.066719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.068861 (+ 2142us) server_negotiation.cc:407] Connection header received
0806 00:48:24.068962 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.068965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.069011 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.069098 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.069580 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57052: BlockingRecv error: recv got EOF from 127.0.0.1:57052 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 00:48:24.069963 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.067179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34792 (local address 127.29.84.189:43347)
0806 00:48:24.067281 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.067284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.069148 (+ 1864us) server_negotiation.cc:407] Connection header received
0806 00:48:24.069287 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.069290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.069329 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.069373 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.069850 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34792: BlockingRecv error: recv got EOF from 127.0.0.1:34792 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 00:48:24.076793 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.075372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34806 (local address 127.29.84.189:43347)
0806 00:48:24.075484 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.075488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.075909 (+ 421us) server_negotiation.cc:407] Connection header received
0806 00:48:24.076119 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.076122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.076185 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.076235 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.076697 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34806: BlockingRecv error: recv got EOF from 127.0.0.1:34806 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 00:48:24.081259 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.080041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34812 (local address 127.29.84.189:43347)
0806 00:48:24.080163 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.080167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.080480 (+ 313us) server_negotiation.cc:407] Connection header received
0806 00:48:24.080642 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.080645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.080686 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.080730 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.081167 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34812: BlockingRecv error: recv got EOF from 127.0.0.1:34812 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 00:48:24.086822 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.085469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.29.84.189:43347)
0806 00:48:24.085572 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.085575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.086038 (+ 463us) server_negotiation.cc:407] Connection header received
0806 00:48:24.086205 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.086208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.086247 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.086294 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.086740 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34824: BlockingRecv error: recv got EOF from 127.0.0.1:34824 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 00:48:24.094322 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.092887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.29.84.189:43347)
0806 00:48:24.093027 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.093033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.093501 (+ 468us) server_negotiation.cc:407] Connection header received
0806 00:48:24.093681 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.093684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.093725 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.093772 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.094244 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34834: BlockingRecv error: recv got EOF from 127.0.0.1:34834 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 00:48:24.105875 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.104424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34840 (local address 127.29.84.189:43347)
0806 00:48:24.104512 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.104515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.104996 (+ 481us) server_negotiation.cc:407] Connection header received
0806 00:48:24.105189 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.105192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.105236 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.105286 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.105797 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34840: BlockingRecv error: recv got EOF from 127.0.0.1:34840 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 00:48:24.125659 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.124131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34848 (local address 127.29.84.189:43347)
0806 00:48:24.124219 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.124223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.124756 (+ 533us) server_negotiation.cc:407] Connection header received
0806 00:48:24.124928 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.124932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.124985 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.125029 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.125546 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34848: BlockingRecv error: recv got EOF from 127.0.0.1:34848 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 00:48:24.161280 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.160109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.29.84.189:43347)
0806 00:48:24.160255 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.160259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.160520 (+ 261us) server_negotiation.cc:407] Connection header received
0806 00:48:24.160603 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.160605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.160652 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.160723 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.161166 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34856: BlockingRecv error: recv got EOF from 127.0.0.1:34856 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 00:48:24.228515 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.227534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.29.84.189:43347)
0806 00:48:24.227662 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.227665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.227819 (+ 154us) server_negotiation.cc:407] Connection header received
0806 00:48:24.227930 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.227932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.227977 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.228051 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.228418 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34870: BlockingRecv error: recv got EOF from 127.0.0.1:34870 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 00:48:24.359751 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.358591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34874 (local address 127.29.84.189:43347)
0806 00:48:24.358737 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.358740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.358915 (+ 175us) server_negotiation.cc:407] Connection header received
0806 00:48:24.359052 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.359055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.359140 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.359219 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.359654 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34874: BlockingRecv error: recv got EOF from 127.0.0.1:34874 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 00:48:24.618755 31193 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:24.617735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.29.84.189:43347)
0806 00:48:24.617866 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:24.617870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:24.617995 (+ 125us) server_negotiation.cc:407] Connection header received
0806 00:48:24.618109 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:24.618112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:24.618158 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:24.618226 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:24.618661 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34882: BlockingRecv error: recv got EOF from 127.0.0.1:34882 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:48:24.920156 31359 catalog_manager.cc:1259] Loaded cluster ID: 1a3fe055ce43469f86ea9ca2377013fe
I20260806 00:48:24.920267 31359 catalog_manager.cc:1552] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98: loading cluster ID for follower catalog manager: success
I20260806 00:48:24.921264 31324 catalog_manager.cc:1259] Loaded cluster ID: 1a3fe055ce43469f86ea9ca2377013fe
I20260806 00:48:24.921337 31324 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297: loading cluster ID for follower catalog manager: success
I20260806 00:48:24.922591 31359 catalog_manager.cc:1574] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98: acquiring CA information for follower catalog manager: success
I20260806 00:48:24.923319 31324 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297: acquiring CA information for follower catalog manager: success
I20260806 00:48:24.923982 31359 catalog_manager.cc:1602] T 00000000000000000000000000000000 P afb335cd15fa43b69b377e832b009a98: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:48:24.924435 31324 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a2114a3b3ece47bd9b3d5a6b257df297: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:48:25.136322 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:25.132756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34884 (local address 127.29.84.189:43347)
0806 00:48:25.132996 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:25.132999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:25.133068 (+ 69us) server_negotiation.cc:407] Connection header received
0806 00:48:25.133188 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:25.133190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:25.133315 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:25.133414 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:25.133667 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.134167 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.134981 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.135209 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.135282 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:25.135594 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:25.136011 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:25.136101 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:25.136158 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 00:48:25.136205 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":104,"threads_started":1}
I20260806 00:48:25.142629 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:25.139456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57064 (local address 127.29.84.188:40699)
0806 00:48:25.139658 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:25.139662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:25.139674 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:48:25.139800 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:25.139802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:25.139901 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:25.139989 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:25.140156 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.140667 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.141416 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.141638 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.141729 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:25.141970 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:25.142301 (+ 331us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:25.142385 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:25.142455 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 00:48:25.142497 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260806 00:48:25.149753 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:25.145561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.29.84.189:43347)
0806 00:48:25.145665 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:25.145669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:25.146089 (+ 420us) server_negotiation.cc:407] Connection header received
0806 00:48:25.146275 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:25.146279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:25.146325 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:25.146373 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:25.146798 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.147553 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.148644 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.148869 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.148989 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:25.149268 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:25.149337 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:25.149434 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:25.149570 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 00:48:25.149631 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:48:25.150076 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:25.146242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57078 (local address 127.29.84.188:40699)
0806 00:48:25.146348 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:25.146355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:25.146530 (+ 175us) server_negotiation.cc:407] Connection header received
0806 00:48:25.146632 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:25.146635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:25.146680 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:25.146784 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:25.147029 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.147556 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.148685 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.148998 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.149110 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:25.149353 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:25.149512 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:25.149582 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:25.149907 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 00:48:25.149955 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 00:48:25.150936 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:25.145485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43042 (local address 127.29.84.190:44819)
0806 00:48:25.146396 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:25.146400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:25.146414 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:48:25.146503 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:25.146505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:25.146653 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:25.147151 (+ 498us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:25.147261 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.148007 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.149091 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:25.149480 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:25.149905 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:25.150033 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:25.150319 (+ 286us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:25.150700 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:25.150737 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 00:48:25.150798 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":825,"thread_start_us":84,"threads_started":1}
I20260806 00:48:25.153820 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 30975
I20260806 00:48:25.164019 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 31085
I20260806 00:48:25.175885 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 31194
2026-08-06T00:48:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3254 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:25 dist-test-slave-n326 krb5kdc[31515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:25 dist-test-slave-n326 krb5kdc[31515](info): set up 2 sockets
Aug 06 00:48:25 dist-test-slave-n326 krb5kdc[31515](info): commencing operation
krb5kdc: starting...
W20260806 00:48:26.408502 30034 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/test-user.keytab.
Aug 06 00:48:26 dist-test-slave-n326 krb5kdc[31515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.84.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
Entry for principal kudu/127.29.84.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab.
I20260806 00:48:26.530264 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:41493
--webserver_interface=127.29.84.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:41493,127.29.84.189:46559,127.29.84.188:34249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:26.637849 31531 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:48:26.638088 31531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:26.638113 31531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:26.638147 31531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:26.641950 31531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:26.641996 31531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:26.642015 31531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:26.642030 31531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:26.642045 31531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:26.647050 31531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:41493,127.29.84.189:46559,127.29.84.188:34249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.190.keytab
--principal=kudu/127.29.84.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.190:41493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.84.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:26.647905 31531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:26.648743 31531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:26.656456 31541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:26.656502 31537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:26.656466 31538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:26.656770 31531 server_base.cc:1034] running on GCE node
I20260806 00:48:26.657303 31531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:26.658217 31531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:26.659420 31531 hybrid_clock.cc:648] HybridClock initialized: now 1785977306659395 us; error 35 us; skew 500 ppm
Aug 06 00:48:26 dist-test-slave-n326 krb5kdc[31515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:26.661999 31531 init.cc:377] Logged in from keytab as kudu/127.29.84.190@KRBTEST.COM (short username kudu)
I20260806 00:48:26.662861 31531 webserver.cc:460] Webserver started at http://127.29.84.190:36329/ using document root <none> and password file <none>
I20260806 00:48:26.663437 31531 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:26.663527 31531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:26.663753 31531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:26.665647 31531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/data/instance:
uuid: "07ea9fb540f949719e8040784dd03340"
format_stamp: "Formatted at 2026-08-06 00:48:26 on dist-test-slave-n326"
server_key: "c294cf2c42264891371dda364225b234"
server_key_iv: "5d2eb00baa8ead8adfe9270f55241fa1"
server_key_version: "encryptionkey@0"
I20260806 00:48:26.666194 31531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/wal/instance:
uuid: "07ea9fb540f949719e8040784dd03340"
format_stamp: "Formatted at 2026-08-06 00:48:26 on dist-test-slave-n326"
server_key: "c294cf2c42264891371dda364225b234"
server_key_iv: "5d2eb00baa8ead8adfe9270f55241fa1"
server_key_version: "encryptionkey@0"
I20260806 00:48:26.669593 31531 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:48:26.671840 31548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:26.672946 31531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:26.673084 31531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/wal
uuid: "07ea9fb540f949719e8040784dd03340"
format_stamp: "Formatted at 2026-08-06 00:48:26 on dist-test-slave-n326"
server_key: "c294cf2c42264891371dda364225b234"
server_key_iv: "5d2eb00baa8ead8adfe9270f55241fa1"
server_key_version: "encryptionkey@0"
I20260806 00:48:26.673188 31531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:26.683976 31531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:26.686579 31531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:26.686767 31531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:26.693791 31531 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.190:41493
I20260806 00:48:26.694082 31630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.190:41493 every 8 connection(s)
I20260806 00:48:26.694697 31531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:26.695523 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 31531
I20260806 00:48:26.695664 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-0/wal/instance
I20260806 00:48:26.695959 30034 external_mini_cluster.cc:1428] Setting key e8bee506680c62bb1d37f01c680f981e
I20260806 00:48:26.697427 31631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:26.701337 31631 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.84.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.189@KRBTEST.COM" created.
Aug 06 00:48:26 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:48:26.716117 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.703545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:41493 (local address 127.0.0.1:54702)
0806 00:48:26.704040 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.704090 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:26.704947 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:26.705604 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:26.705609 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:26.706015 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:26.706240 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:26.706252 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.707449 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.707454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:26.708237 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:26.708246 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.708850 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.708854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:26.709036 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:26.709588 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:26.709604 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:26.711461 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:26.713389 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:26.713397 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:26.713405 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:26.713633 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:26.713946 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:26.713949 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:26.713951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:26.714063 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:26.714607 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:26.714612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:26.714742 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 00:48:26.714902 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 00:48:26.715217 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":57,"thread_start_us":149,"threads_started":1}
I20260806 00:48:26.716117 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.703544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.29.84.190:41493)
0806 00:48:26.704057 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:26.704075 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:26.704205 (+ 130us) server_negotiation.cc:407] Connection header received
0806 00:48:26.705233 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:26.705239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:26.705422 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:26.705604 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:26.706384 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.707325 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.708357 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.708704 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.708927 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:26.711633 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:26.711639 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:26.711650 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:26.711673 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:26.713261 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:26.713740 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:26.713745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:26.713747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:26.713849 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:26.714159 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:26.714164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:26.714166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:26.714487 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:26.714619 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:26.714960 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 00:48:26.715268 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":417,"spinlock_wait_cycles":53504,"thread_start_us":180,"threads_started":1}
I20260806 00:48:26.718546 31631 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:48:26.719255 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.718955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:46559 (local address 127.0.0.1:53628)
0806 00:48:26.719104 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.719186 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 00:48:26.719461 31551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
I20260806 00:48:26.720284 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.720064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:46559 (local address 127.0.0.1:53630)
0806 00:48:26.720155 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.720196 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 00:48:26.720516 31631 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.189:46559: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab.
Entry for principal kudu/127.29.84.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab.
I20260806 00:48:26.723554 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.189:46559
--webserver_interface=127.29.84.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab
--principal=kudu/127.29.84.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:41493,127.29.84.189:46559,127.29.84.188:34249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:48:26.769766 31631 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } attempt: 1
I20260806 00:48:26.770550 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.770260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:46559 (local address 127.0.0.1:53634)
0806 00:48:26.770416 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.770472 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 00:48:26.771274 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.771035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:46559 (local address 127.0.0.1:53642)
0806 00:48:26.771171 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.771221 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 00:48:26.771396 31631 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.189:46559: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
W20260806 00:48:26.829556 31643 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:48:26.829833 31643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:26.829916 31643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:26.829993 31643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:26.833818 31643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:26.833890 31643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:26.833925 31643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:26.833997 31643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:26.834035 31643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:26.838858 31643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:41493,127.29.84.189:46559,127.29.84.188:34249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.189.keytab
--principal=kudu/127.29.84.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.189:46559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.84.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:26.839546 31631 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } attempt: 2
I20260806 00:48:26.839799 31643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:26.840302 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.840037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:46559 (local address 127.0.0.1:53656)
0806 00:48:26.840171 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.840236 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 00:48:26.840746 31643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:48:26.840976 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.840769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:46559 (local address 127.0.0.1:53658)
0806 00:48:26.840875 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.840921 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:48:26.841113 31631 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.189:46559: Network error: Client connection negotiation failed: client connection to 127.29.84.189:46559: connect: Connection refused (error 111)
W20260806 00:48:26.848517 31655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:26.848558 31651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:26.848543 31650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:26.848805 31643 server_base.cc:1034] running on GCE node
I20260806 00:48:26.849313 31643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:26.850401 31643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:26.851599 31643 hybrid_clock.cc:648] HybridClock initialized: now 1785977306851565 us; error 43 us; skew 500 ppm
Aug 06 00:48:26 dist-test-slave-n326 krb5kdc[31515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:26.854076 31643 init.cc:377] Logged in from keytab as kudu/127.29.84.189@KRBTEST.COM (short username kudu)
I20260806 00:48:26.854849 31643 webserver.cc:460] Webserver started at http://127.29.84.189:34773/ using document root <none> and password file <none>
I20260806 00:48:26.855371 31643 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:26.855456 31643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:26.855659 31643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:26.857373 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/data/instance:
uuid: "af877ec74179429090c3bfda13f9a4c5"
format_stamp: "Formatted at 2026-08-06 00:48:26 on dist-test-slave-n326"
server_key: "8c82f1f5c0adbd6109da5ef272e0e904"
server_key_iv: "2e3414f062edccc5e60b741d773e3c9f"
server_key_version: "encryptionkey@0"
I20260806 00:48:26.857892 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/wal/instance:
uuid: "af877ec74179429090c3bfda13f9a4c5"
format_stamp: "Formatted at 2026-08-06 00:48:26 on dist-test-slave-n326"
server_key: "8c82f1f5c0adbd6109da5ef272e0e904"
server_key_iv: "2e3414f062edccc5e60b741d773e3c9f"
server_key_version: "encryptionkey@0"
I20260806 00:48:26.861296 31643 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:48:26.863533 31664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:26.864585 31643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:26.864719 31643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/wal
uuid: "af877ec74179429090c3bfda13f9a4c5"
format_stamp: "Formatted at 2026-08-06 00:48:26 on dist-test-slave-n326"
server_key: "8c82f1f5c0adbd6109da5ef272e0e904"
server_key_iv: "2e3414f062edccc5e60b741d773e3c9f"
server_key_version: "encryptionkey@0"
I20260806 00:48:26.864820 31643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:26.880815 31643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:26.883319 31643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:26.883500 31643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:26.889816 31643 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.189:46559
I20260806 00:48:26.890076 31740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.189:46559 every 8 connection(s)
I20260806 00:48:26.890678 31643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/data/info.pb
I20260806 00:48:26.893103 31741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:26.896555 31741 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:48:26.899313 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 31643
I20260806 00:48:26.899449 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-1/wal/instance
I20260806 00:48:26.899782 30034 external_mini_cluster.cc:1428] Setting key a6a8dbdfea87974b23f074d858cac32e
Aug 06 00:48:26 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.84.188@KRBTEST.COM; defaulting to no policy
I20260806 00:48:26.911090 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.898143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.29.84.190:41493)
0806 00:48:26.898280 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:26.898286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:26.899099 (+ 813us) server_negotiation.cc:407] Connection header received
0806 00:48:26.900243 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:26.900245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:26.900293 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:26.900367 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:26.901879 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.902771 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.904374 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.904613 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.904697 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:26.907481 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:26.907484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:26.907486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:26.907509 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:26.908922 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:26.909464 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:26.909467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:26.909469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:26.909550 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:26.909887 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:26.909891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:26.909892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:26.910160 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:26.910282 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:26.910774 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 00:48:26.910894 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:48:26.911728 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.898517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:41493 (local address 127.0.0.1:54704)
0806 00:48:26.898916 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.898954 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:26.900033 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:26.900564 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:26.900575 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:26.901076 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:26.901718 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:26.901739 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.902897 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.902899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:26.904228 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:26.904242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.904783 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.904791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:26.905038 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:26.905625 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:26.905644 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:26.907299 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:26.909075 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:26.909081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:26.909093 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:26.909353 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:26.909649 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:26.909652 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:26.909654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:26.909784 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:26.910280 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:26.910287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:26.910526 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 00:48:26.910699 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 00:48:26.910986 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.84.188@KRBTEST.COM" created.
I20260806 00:48:26.913045 31741 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:48:26 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:48:26.922282 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.913524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:46559 (local address 127.0.0.1:53672)
0806 00:48:26.913690 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.913705 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:26.913776 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:26.914323 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:26.914325 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:26.914443 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:26.914540 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:26.914545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.916020 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.916023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:26.916517 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:26.916524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.917354 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.917357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:26.917467 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:26.917910 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:26.917920 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:26.918988 (+ 1068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:26.920842 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:26.920848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:26.920852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:26.921045 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:26.921341 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:26.921345 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:26.921347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:26.921439 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:26.921924 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:26.921926 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:26.922000 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 00:48:26.922089 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 00:48:26.922179 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":20}
I20260806 00:48:26.922513 31752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.913584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53672 (local address 127.29.84.189:46559)
0806 00:48:26.913920 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:26.913934 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:26.913958 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:48:26.914004 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:26.914009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:26.914196 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:26.914333 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:26.914653 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.915895 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.916637 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.917242 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.917388 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:26.919156 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:26.919169 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:26.919183 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:26.919207 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:26.920732 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:26.921150 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:26.921156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:26.921158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:26.921234 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:26.921546 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:26.921549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:26.921551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:26.921758 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:26.921950 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:26.922207 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 00:48:26.922330 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":228,"thread_start_us":136,"threads_started":1}
I20260806 00:48:26.924019 31741 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:48:26.924625 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.924390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41106)
0806 00:48:26.924494 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.924576 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 00:48:26.924849 31665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
I20260806 00:48:26.925660 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.925461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41118)
0806 00:48:26.925564 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.925614 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:48:26.925820 31741 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:34249: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.84.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab.
Entry for principal kudu/127.29.84.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab.
I20260806 00:48:26.928381 30034 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
/tmp/dist-test-taskdVRgL9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.188:34249
--webserver_interface=127.29.84.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab
--principal=kudu/127.29.84.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.84.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.84.190:41493,127.29.84.189:46559,127.29.84.188:34249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:48:26.932292 31631 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } attempt: 3
Aug 06 00:48:26 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:48:26.940593 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.932733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:46559 (local address 127.0.0.1:53676)
0806 00:48:26.932850 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.932865 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:26.932939 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:26.933154 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:26.933156 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:26.933289 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:26.933375 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:26.933379 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.934435 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.934437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:26.934952 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:26.934960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.935671 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.935673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:26.935752 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:26.936200 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:26.936212 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:26.937365 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:26.939314 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:26.939316 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:26.939317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:26.939524 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:26.939788 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:26.939790 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:26.939791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:26.939862 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:26.940221 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:26.940223 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:26.940296 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 00:48:26.940379 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 00:48:26.940472 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 00:48:26.940718 31752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.932799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53676 (local address 127.29.84.189:46559)
0806 00:48:26.932929 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:26.932936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:26.932952 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:48:26.932993 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:26.932998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:26.933046 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:26.933142 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:26.933495 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.934302 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.935108 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:26.935553 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:26.935687 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:26.937508 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:26.937515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:26.937521 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:26.937549 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:26.939218 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:26.939614 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:26.939617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:26.939619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:26.939699 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:26.939943 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:26.939946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:26.939948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:26.940133 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:26.940226 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:26.940450 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 00:48:26.940567 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:48:26.941660 31631 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:48:26.942296 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.942046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41120)
0806 00:48:26.942174 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.942228 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 00:48:26.942916 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.942675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41126)
0806 00:48:26.942814 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.942862 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260806 00:48:26.943048 31631 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:34249: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
I20260806 00:48:26.974228 31631 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } attempt: 1
I20260806 00:48:26.974896 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.974633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41134)
0806 00:48:26.974766 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.974825 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 00:48:26.975061 31741 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } attempt: 1
I20260806 00:48:26.975647 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.975423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41140)
0806 00:48:26.975549 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.975596 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 00:48:26.975781 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.975506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41148)
0806 00:48:26.975665 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.975712 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260806 00:48:26.975908 31631 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:34249: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
I20260806 00:48:26.976477 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:26.976287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41156)
0806 00:48:26.976385 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:26.976428 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:48:26.976657 31741 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:34249: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
W20260806 00:48:27.039471 31754 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:48:27.039701 31754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:27.039778 31754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:27.039868 31754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:27.044457 31754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:27.044507 31754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:27.044526 31754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:27.044543 31754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:27.044595 31754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:27.044814 31741 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } attempt: 2
I20260806 00:48:27.045519 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.045253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41172)
0806 00:48:27.045383 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:27.045448 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 00:48:27.046250 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.046047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41180)
0806 00:48:27.046152 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:27.046202 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:48:27.046396 31741 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:34249: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
I20260806 00:48:27.049851 31754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.84.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.84.190:41493,127.29.84.189:46559,127.29.84.188:34249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/krb5kdc/kudu_127.29.84.188.keytab
--principal=kudu/127.29.84.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.84.188:34249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.84.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:27.050761 31754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:27.051100 31631 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } attempt: 2
I20260806 00:48:27.051769 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.051518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41186)
0806 00:48:27.051662 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:27.051708 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 00:48:27.051919 31754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:48:27.052392 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.052188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41200)
0806 00:48:27.052290 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:27.052339 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:48:27.052539 31631 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.84.188:34249: Network error: Client connection negotiation failed: client connection to 127.29.84.188:34249: connect: Connection refused (error 111)
W20260806 00:48:27.059513 31765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:27.059537 31769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:27.059628 31754 server_base.cc:1034] running on GCE node
W20260806 00:48:27.059532 31763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:27.060271 31754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:27.061236 31754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:27.062426 31754 hybrid_clock.cc:648] HybridClock initialized: now 1785977307062413 us; error 23 us; skew 500 ppm
Aug 06 00:48:27 dist-test-slave-n326 krb5kdc[31515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:27.065140 31754 init.cc:377] Logged in from keytab as kudu/127.29.84.188@KRBTEST.COM (short username kudu)
I20260806 00:48:27.065847 31754 webserver.cc:460] Webserver started at http://127.29.84.188:32885/ using document root <none> and password file <none>
I20260806 00:48:27.066303 31754 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:27.066355 31754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:27.066524 31754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:27.068413 31754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/data/instance:
uuid: "7af34f741496494dbd3c4c92b8e1fe87"
format_stamp: "Formatted at 2026-08-06 00:48:27 on dist-test-slave-n326"
server_key: "1d22d30ec3602ef15fb5ef5f5024e6ae"
server_key_iv: "81a3612e74b70c145945437cd3d58604"
server_key_version: "encryptionkey@0"
I20260806 00:48:27.068926 31754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/wal/instance:
uuid: "7af34f741496494dbd3c4c92b8e1fe87"
format_stamp: "Formatted at 2026-08-06 00:48:27 on dist-test-slave-n326"
server_key: "1d22d30ec3602ef15fb5ef5f5024e6ae"
server_key_iv: "81a3612e74b70c145945437cd3d58604"
server_key_version: "encryptionkey@0"
I20260806 00:48:27.072350 31754 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:48:27.074636 31781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:27.075819 31754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:27.075963 31754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/data,/tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/wal
uuid: "7af34f741496494dbd3c4c92b8e1fe87"
format_stamp: "Formatted at 2026-08-06 00:48:27 on dist-test-slave-n326"
server_key: "1d22d30ec3602ef15fb5ef5f5024e6ae"
server_key_iv: "81a3612e74b70c145945437cd3d58604"
server_key_version: "encryptionkey@0"
I20260806 00:48:27.076122 31754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:27.093192 31754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:27.095899 31754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:27.096083 31754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:27.102581 31754 rpc_server.cc:307] RPC server started. Bound to: 127.29.84.188:34249
I20260806 00:48:27.102928 31863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.84.188:34249 every 8 connection(s)
I20260806 00:48:27.103538 31754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/data/info.pb
I20260806 00:48:27.103770 30034 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu as pid 31754
I20260806 00:48:27.103909 30034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdVRgL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977020829091-30034-0/minicluster-data/master-2/wal/instance
I20260806 00:48:27.104295 30034 external_mini_cluster.cc:1428] Setting key 3708f924e94a04db759fc5757a0ecc84
I20260806 00:48:27.106549 31864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:27.110811 31864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:48:27 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
Aug 06 00:48:27 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
I20260806 00:48:27.118641 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.105559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41208 (local address 127.29.84.188:34249)
0806 00:48:27.105934 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.105949 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.105982 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:48:27.106581 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.106624 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.106952 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.107225 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.108097 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.109024 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.110221 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.110664 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.110775 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.113863 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.113869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.113884 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.113912 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.115662 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.116102 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.116113 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.116117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.116224 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.116508 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.116512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.116516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.116923 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.117140 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.117435 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.117742 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":132,"threads_started":1}
I20260806 00:48:27.122987 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.112911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.190:41493 (local address 127.0.0.1:54720)
0806 00:48:27.113340 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:27.113379 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:27.113520 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:27.114177 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:27.114182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:27.114504 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:27.114612 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.114625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.115442 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.115445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.116041 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.116050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.116516 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.116519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.116656 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.117186 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:27.117201 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:27.118829 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:27.121412 (+ 2583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.121420 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.121424 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.121635 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.121932 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.121935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.121937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.122035 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.122400 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:27.122406 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:27.122523 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 00:48:27.122671 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 00:48:27.122843 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":153,"threads_started":1}
I20260806 00:48:27.123049 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.112957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.29.84.190:41493)
0806 00:48:27.113081 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.113085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.113468 (+ 383us) server_negotiation.cc:407] Connection header received
0806 00:48:27.113619 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.113622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.113676 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.113768 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.114748 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.115323 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.116185 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.116398 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.116500 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.118970 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.118974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.118976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.118998 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.121295 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.121735 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.121738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.121740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.121830 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.122131 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.122134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.122135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.122294 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.122371 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.122773 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.122893 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:48:27.124152 31864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:48:27 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.190@KRBTEST.COM
Aug 06 00:48:27 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:48:27.130430 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.120781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54722 (local address 127.29.84.190:41493)
0806 00:48:27.121070 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.121077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.121088 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:48:27.121115 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.121117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.121147 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.121216 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.121923 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.122653 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.123668 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.123907 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.124024 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.126845 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.126848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.126850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.126871 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.128567 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.128972 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.128978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.128983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.129075 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.129379 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.129381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.129382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.129638 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.129834 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.130062 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.130183 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":118,"threads_started":1}
I20260806 00:48:27.132864 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.124635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.189:46559 (local address 127.0.0.1:53678)
0806 00:48:27.124790 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:27.124804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:27.124881 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:27.125124 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:27.125129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:27.125317 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:27.125436 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.125442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.126421 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.126424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.127117 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.127133 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.127621 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.127624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.127722 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.128092 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:27.128103 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:27.129338 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:27.131460 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.131463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.131464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.131730 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.132016 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.132020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.132025 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.132119 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.132535 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:27.132537 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:27.132604 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 00:48:27.132684 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 00:48:27.132768 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 00:48:27.133052 31752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.124740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53678 (local address 127.29.84.189:46559)
0806 00:48:27.124871 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.124874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.124887 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:48:27.124974 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.124976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.125014 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.125079 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.125551 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.126316 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.127263 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.127498 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.127609 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.129473 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.129476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.129479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.129505 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.131351 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.131846 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.131853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.131855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.131927 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.132218 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.132221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.132222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.132425 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.132527 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.132781 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.132898 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:48:27.133610 31864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:48:27 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.84.189@KRBTEST.COM
I20260806 00:48:27.137555 31741 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } attempt: 3
Aug 06 00:48:27 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.188@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
I20260806 00:48:27.141227 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.131609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53688 (local address 127.29.84.189:46559)
0806 00:48:27.132212 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.132224 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.132243 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:48:27.132271 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.132273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.132304 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.132365 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.133091 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.133775 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.134718 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.135023 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.135157 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.137624 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.137627 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.137629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.137655 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.139367 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.139811 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.139814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.139816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.139926 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.140285 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.140288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.140290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.140518 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.140699 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.140907 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.141021 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":516,"thread_start_us":100,"threads_started":1}
I20260806 00:48:27.142400 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.134051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41222)
0806 00:48:27.134221 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:27.134232 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:27.134285 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:27.134463 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:27.134465 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:27.134640 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:27.134735 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.134742 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.136089 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.136092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.136588 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.136599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.137050 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.137057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.137143 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.137517 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:27.137535 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:27.138997 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:27.140787 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.140790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.140792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.141030 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.141308 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.141312 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.141316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.141419 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.141878 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:27.141881 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:27.141973 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 00:48:27.142086 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 00:48:27.142215 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 00:48:27.142419 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.134158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41222 (local address 127.29.84.188:34249)
0806 00:48:27.134285 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.134290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.134303 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:48:27.134336 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.134339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.134374 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.134451 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.134849 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.135971 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.136712 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.136930 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.137007 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.139161 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.139164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.139167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.139189 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.140693 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.141130 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.141133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.141134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.141210 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.141518 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.141523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.141524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.141720 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.141841 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.142140 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.142271 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":53}
Aug 06 00:48:27 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.189@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
I20260806 00:48:27.146087 31864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87: Bootstrap starting.
I20260806 00:48:27.146245 30034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:27.147490 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.137899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41232)
0806 00:48:27.137990 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:27.138004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:27.138085 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:27.138859 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:27.138862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:27.139109 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:27.139221 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.139230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.140285 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.140288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.141129 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.141139 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.141826 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.141830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.141919 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.142372 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:27.142390 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:27.143874 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:27.145925 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.145931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.145933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.146209 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.146517 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.146520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.146522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.146605 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.147104 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:27.147107 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:27.147207 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 00:48:27.147295 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 00:48:27.147389 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":40}
I20260806 00:48:27.147967 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.137979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41232 (local address 127.29.84.188:34249)
0806 00:48:27.138302 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.138310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.138321 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:48:27.138350 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.138352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.138650 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.138826 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.139365 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.140176 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.141252 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.141729 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.141827 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.144021 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.144025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.144027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.144050 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.145821 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.146326 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.146329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.146331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.146408 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.146717 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.146721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.146723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.146959 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.147680 (+ 721us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.147760 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.147844 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":128,"threads_started":1}
I20260806 00:48:27.148850 31864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:27.147596 31593 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "07ea9fb540f949719e8040784dd03340"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54722
I20260806 00:48:27.149744 31864 log.cc:824] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:27.151686 31631 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } attempt: 3
I20260806 00:48:27.151712 31741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5: Bootstrap starting.
I20260806 00:48:27.152842 31864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87: No bootstrap required, opened a new log
I20260806 00:48:27.153925 31741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:27.154610 31741 log.cc:824] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:27.156287 31741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5: No bootstrap required, opened a new log
I20260806 00:48:27.156749 31864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } }
I20260806 00:48:27.156973 31864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af34f741496494dbd3c4c92b8e1fe87, State: Initialized, Role: FOLLOWER
I20260806 00:48:27.157292 31864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } }
I20260806 00:48:27.157775 31878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.157918 31878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:27.158556 31864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:27.159495 31741 raft_consensus.cc:348] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } }
I20260806 00:48:27.159730 31741 raft_consensus.cc:729] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af877ec74179429090c3bfda13f9a4c5, State: Initialized, Role: FOLLOWER
I20260806 00:48:27.159955 31741 consensus_queue.cc:260] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } }
I20260806 00:48:27.160341 31887 sys_catalog.cc:455] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.160488 31887 sys_catalog.cc:458] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:27.161140 31741 sys_catalog.cc:564] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 00:48:27.163967 31900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:27.164050 31900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 00:48:27 dist-test-slave-n326 krb5kdc[31515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.84.190@KRBTEST.COM for kudu/127.29.84.188@KRBTEST.COM
I20260806 00:48:27.162367 31754 master_runner.cc:186] Error getting master registration for 127.29.84.190:41493: OK, instance_id { permanent_uuid: "07ea9fb540f949719e8040784dd03340" instance_seqno: 1785977306691982 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:48:27.166205 31754 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 00:48:27.166426 31918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:27.166498 31918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:48:27.166844 31643 master_runner.cc:186] Error getting master registration for 127.29.84.190:41493: OK, instance_id { permanent_uuid: "07ea9fb540f949719e8040784dd03340" instance_seqno: 1785977306691982 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:48:27.171536 31643 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:48:27.171715 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.152027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.84.188:34249 (local address 127.0.0.1:41246)
0806 00:48:27.152175 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:27.152188 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:27.152265 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:27.159619 (+ 7354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:27.159623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:27.159817 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:27.159920 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.159927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.162301 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.162305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.163121 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:27.163132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.164026 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.164032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:27.164120 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.164606 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:27.164620 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:27.166781 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:27.169684 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.169687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.169689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.169925 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.170407 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:27.170422 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:27.170424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:27.170520 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:27.171182 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:27.171186 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:27.171295 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 00:48:27.171388 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:48:27.171529 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 00:48:27.172369 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.152081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.29.84.188:34249)
0806 00:48:27.159204 (+ 7123us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.159209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.159251 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:48:27.159318 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.159322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.159404 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.160170 (+ 766us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.160392 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.162122 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.163303 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.163821 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.164383 (+ 562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.166977 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.166981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.166984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.167048 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.169537 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.170115 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.170119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.170121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.170251 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.170676 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.170679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.170681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.170992 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.172036 (+ 1044us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.172111 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.172226 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7030}
I20260806 00:48:27.174685 31631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340: Bootstrap starting.
I20260806 00:48:27.177160 31631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:27.177966 31631 log.cc:824] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:27.179975 31631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340: No bootstrap required, opened a new log
I20260806 00:48:27.182878 31631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } }
I20260806 00:48:27.183131 31631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07ea9fb540f949719e8040784dd03340, State: Initialized, Role: FOLLOWER
I20260806 00:48:27.183442 31631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } }
I20260806 00:48:27.183948 31924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.184198 31924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:27.184713 31631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:27.184798 31593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 00:48:27.184883 31593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:27.186229 31593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } }
I20260806 00:48:27.187937 31698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07ea9fb540f949719e8040784dd03340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "af877ec74179429090c3bfda13f9a4c5"
I20260806 00:48:27.188066 31820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07ea9fb540f949719e8040784dd03340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7af34f741496494dbd3c4c92b8e1fe87"
I20260806 00:48:27.188177 31698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:27.188253 31820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:27.189446 31820 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07ea9fb540f949719e8040784dd03340 in term 1.
I20260806 00:48:27.189446 31698 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07ea9fb540f949719e8040784dd03340 in term 1.
I20260806 00:48:27.187259 31593 leader_election.cc:288] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [CANDIDATE]: Term 1 election: Requested vote from peers af877ec74179429090c3bfda13f9a4c5 (127.29.84.189:46559), 7af34f741496494dbd3c4c92b8e1fe87 (127.29.84.188:34249)
W20260806 00:48:27.189746 31556 tablet.cc:2369] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:48:27.189838 31551 leader_election.cc:302] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07ea9fb540f949719e8040784dd03340, af877ec74179429090c3bfda13f9a4c5; no voters:
I20260806 00:48:27.193337 31924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:27.193619 31924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [term 1 LEADER]: Becoming Leader. State: Replica: 07ea9fb540f949719e8040784dd03340, State: Running, Role: LEADER
I20260806 00:48:27.193965 31924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } }
I20260806 00:48:27.196072 31924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07ea9fb540f949719e8040784dd03340. Latest consensus state: current_term: 1 leader_uuid: "07ea9fb540f949719e8040784dd03340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.196768 31924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:27.196919 31531 master_runner.cc:263] No leader master found from master 07ea9fb540f949719e8040784dd03340
I20260806 00:48:27.197016 31531 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 00:48:27.199545 31946 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:48:27.199676 31946 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:48:27.199772 31961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:27.200660 31961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:27.207165 31820 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [term 1 FOLLOWER]: Refusing update from remote peer 07ea9fb540f949719e8040784dd03340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:27.207854 31924 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:27.211865 31878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07ea9fb540f949719e8040784dd03340. Latest consensus state: current_term: 1 leader_uuid: "07ea9fb540f949719e8040784dd03340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.211977 31878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:27.212965 31698 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [term 1 FOLLOWER]: Refusing update from remote peer 07ea9fb540f949719e8040784dd03340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:27.213544 31924 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:27.216704 31887 sys_catalog.cc:455] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07ea9fb540f949719e8040784dd03340. Latest consensus state: current_term: 1 leader_uuid: "07ea9fb540f949719e8040784dd03340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.216814 31887 sys_catalog.cc:458] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:27.221469 31956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "07ea9fb540f949719e8040784dd03340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.221580 31956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:27.222646 31878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "07ea9fb540f949719e8040784dd03340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.222746 31878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:27.223483 31965 mvcc.cc:205] Tried to move back new op lower bound from 7315363050313531392 to 7315363050271485952. Current Snapshot: MvccSnapshot[applied={T|T < 7315363050313531392}]
I20260806 00:48:27.225142 31924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "07ea9fb540f949719e8040784dd03340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.225239 31924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:27.226229 31887 sys_catalog.cc:455] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "07ea9fb540f949719e8040784dd03340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07ea9fb540f949719e8040784dd03340" member_type: VOTER last_known_addr { host: "127.29.84.190" port: 41493 } } peers { permanent_uuid: "af877ec74179429090c3bfda13f9a4c5" member_type: VOTER last_known_addr { host: "127.29.84.189" port: 46559 } } peers { permanent_uuid: "7af34f741496494dbd3c4c92b8e1fe87" member_type: VOTER last_known_addr { host: "127.29.84.188" port: 34249 } } }
I20260806 00:48:27.226327 31887 sys_catalog.cc:458] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:48:27.226976 31970 mvcc.cc:205] Tried to move back new op lower bound from 7315363050313531392 to 7315363050271485952. Current Snapshot: MvccSnapshot[applied={T|T < 7315363050313531392}]
I20260806 00:48:27.228559 31961 catalog_manager.cc:1347] Generated new cluster ID: b2cd78bfecf04a9aa3e98afa78d7234c
I20260806 00:48:27.228611 31961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:27.262655 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.217642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.29.84.190:41493)
0806 00:48:27.217747 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.217752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.239528 (+ 21776us) server_negotiation.cc:407] Connection header received
0806 00:48:27.239710 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.239713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.239765 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.239857 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.243181 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.244026 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.245219 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.245581 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.245672 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.258646 (+ 12974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.258650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.258652 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.258679 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.260365 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.260923 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.260927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.260929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.261025 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.261336 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.261340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.261342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.261572 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.261709 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.262349 (+ 640us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.262472 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 00:48:27.266505 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.217304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.29.84.189:46559)
0806 00:48:27.217429 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.217434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.247309 (+ 29875us) server_negotiation.cc:407] Connection header received
0806 00:48:27.247775 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.247778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.247824 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.247871 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.249615 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.250366 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.253645 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.254032 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.254105 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.257867 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.257873 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.257877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.257903 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.261179 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.264586 (+ 3407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.264590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.264592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.264674 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.265359 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.265362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.265364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.265564 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.265682 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.266238 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.266357 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 00:48:27.267325 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:27.218385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41258 (local address 127.29.84.188:34249)
0806 00:48:27.218486 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.218490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.247521 (+ 29031us) server_negotiation.cc:407] Connection header received
0806 00:48:27.247991 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.247994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.248053 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.248102 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.250518 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.251292 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.254730 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:27.255102 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:27.255175 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:27.261332 (+ 6157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:27.261336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:27.261338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:27.261364 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:27.263042 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.264197 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.264201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.264203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.264286 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:27.264953 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:27.264956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:27.264958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:27.265167 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:27.265282 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:27.266963 (+ 1681us) server_negotiation.cc:299] Negotiation successful
0806 00:48:27.267169 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 00:48:27.301734 31961 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:27.303027 31961 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:27.325626 31961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07ea9fb540f949719e8040784dd03340: Generated new TSK 0
I20260806 00:48:27.326426 31961 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:48:27.359030 31872 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.357207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54734 (local address 127.29.84.190:41493)
0806 00:48:27.357359 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.357364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.357927 (+ 563us) server_negotiation.cc:407] Connection header received
0806 00:48:27.358147 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.358150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.358201 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.358252 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.358878 (+ 626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54734: BlockingRecv error: recv got EOF from 127.0.0.1:54734 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 00:48:27.361891 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.359918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53698 (local address 127.29.84.189:46559)
0806 00:48:27.360064 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.360069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.360842 (+ 773us) server_negotiation.cc:407] Connection header received
0806 00:48:27.361058 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.361062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.361118 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.361175 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.361758 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53698: BlockingRecv error: recv got EOF from 127.0.0.1:53698 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 00:48:27.363854 31875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.360538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.29.84.188:34249)
0806 00:48:27.360667 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.360672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.362675 (+ 2003us) server_negotiation.cc:407] Connection header received
0806 00:48:27.362854 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.362858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.362907 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.362978 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.363720 (+ 742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41262: BlockingRecv error: recv got EOF from 127.0.0.1:41262 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 00:48:27.370517 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.368782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.29.84.189:46559)
0806 00:48:27.368886 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.368890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.369506 (+ 616us) server_negotiation.cc:407] Connection header received
0806 00:48:27.369739 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.369743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.369822 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.369877 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.370401 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53706: BlockingRecv error: recv got EOF from 127.0.0.1:53706 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 00:48:27.375597 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.374025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53716 (local address 127.29.84.189:46559)
0806 00:48:27.374124 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.374128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.374584 (+ 456us) server_negotiation.cc:407] Connection header received
0806 00:48:27.374790 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.374794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.374844 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.374897 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.375485 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53716: BlockingRecv error: recv got EOF from 127.0.0.1:53716 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 00:48:27.381518 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.380087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53722 (local address 127.29.84.189:46559)
0806 00:48:27.380176 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.380180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.380626 (+ 446us) server_negotiation.cc:407] Connection header received
0806 00:48:27.380811 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.380815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.380861 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.380912 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.381434 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53722: BlockingRecv error: recv got EOF from 127.0.0.1:53722 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 00:48:27.388581 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.387647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.29.84.189:46559)
0806 00:48:27.387775 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.387779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.387925 (+ 146us) server_negotiation.cc:407] Connection header received
0806 00:48:27.388003 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.388006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.388048 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.388127 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.388500 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53732: BlockingRecv error: recv got EOF from 127.0.0.1:53732 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 00:48:27.399237 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.398149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.29.84.189:46559)
0806 00:48:27.398259 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.398265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.398452 (+ 187us) server_negotiation.cc:407] Connection header received
0806 00:48:27.398601 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.398604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.398658 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.398746 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.399151 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53742: BlockingRecv error: recv got EOF from 127.0.0.1:53742 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 00:48:27.417876 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.416766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.29.84.189:46559)
0806 00:48:27.416903 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.416909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.417047 (+ 138us) server_negotiation.cc:407] Connection header received
0806 00:48:27.417193 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.417196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.417254 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.417332 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.417756 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53756: BlockingRecv error: recv got EOF from 127.0.0.1:53756 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 00:48:27.452770 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.451795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.29.84.189:46559)
0806 00:48:27.451916 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.451919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.452063 (+ 144us) server_negotiation.cc:407] Connection header received
0806 00:48:27.452155 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.452158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.452197 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.452268 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.452644 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53766: BlockingRecv error: recv got EOF from 127.0.0.1:53766 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 00:48:27.520292 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.519047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53776 (local address 127.29.84.189:46559)
0806 00:48:27.519205 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.519209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.519362 (+ 153us) server_negotiation.cc:407] Connection header received
0806 00:48:27.519533 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.519536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.519592 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.519693 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.520173 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53776: BlockingRecv error: recv got EOF from 127.0.0.1:53776 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 00:48:27.651654 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.650332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.29.84.189:46559)
0806 00:48:27.650518 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.650523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.650662 (+ 139us) server_negotiation.cc:407] Connection header received
0806 00:48:27.650837 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.650841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.650902 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.651002 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.651505 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53790: BlockingRecv error: recv got EOF from 127.0.0.1:53790 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 00:48:27.910802 31873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:27.909775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.29.84.189:46559)
0806 00:48:27.909926 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:27.909930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:27.909998 (+ 68us) server_negotiation.cc:407] Connection header received
0806 00:48:27.910117 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:27.910119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:27.910167 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:27.910237 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:27.910690 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53802: BlockingRecv error: recv got EOF from 127.0.0.1:53802 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:48:28.165309 31900 catalog_manager.cc:1259] Loaded cluster ID: b2cd78bfecf04a9aa3e98afa78d7234c
I20260806 00:48:28.165413 31900 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87: loading cluster ID for follower catalog manager: success
I20260806 00:48:28.167420 31900 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87: acquiring CA information for follower catalog manager: success
I20260806 00:48:28.167853 31918 catalog_manager.cc:1259] Loaded cluster ID: b2cd78bfecf04a9aa3e98afa78d7234c
I20260806 00:48:28.167933 31918 catalog_manager.cc:1552] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5: loading cluster ID for follower catalog manager: success
I20260806 00:48:28.168879 31900 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7af34f741496494dbd3c4c92b8e1fe87: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:48:28.169608 31918 catalog_manager.cc:1574] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5: acquiring CA information for follower catalog manager: success
I20260806 00:48:28.170564 31918 catalog_manager.cc:1602] T 00000000000000000000000000000000 P af877ec74179429090c3bfda13f9a4c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:48:28.428797 32065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:28.425054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.29.84.189:46559)
0806 00:48:28.425322 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:28.425326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:28.425341 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:48:28.425441 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:28.425444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:28.425576 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:28.425689 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:28.425934 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.426506 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.427366 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.427614 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.427701 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:28.427980 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:28.428469 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:28.428571 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:28.428620 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 00:48:28.428670 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":123,"threads_started":1}
I20260806 00:48:28.435297 32079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:28.431929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.29.84.188:34249)
0806 00:48:28.432197 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:28.432200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:28.432213 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:48:28.432287 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:28.432289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:28.432393 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:28.432497 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:28.432670 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.433235 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.433956 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.434211 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.434292 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:28.434530 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:28.434903 (+ 373us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:28.434998 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:28.435053 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 00:48:28.435172 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":111,"threads_started":1}
I20260806 00:48:28.442015 32065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:28.438170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53810 (local address 127.29.84.189:46559)
0806 00:48:28.438493 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:28.438501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:28.438581 (+ 80us) server_negotiation.cc:407] Connection header received
0806 00:48:28.438776 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:28.438781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:28.438835 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:28.438882 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:28.439244 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.439813 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.440901 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.441152 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.441259 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:28.441508 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:28.441580 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:28.441659 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:28.441814 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 00:48:28.441860 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260806 00:48:28.443156 32079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:28.438703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41274 (local address 127.29.84.188:34249)
0806 00:48:28.438814 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:28.438818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:28.439100 (+ 282us) server_negotiation.cc:407] Connection header received
0806 00:48:28.439255 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:28.439258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:28.439304 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:28.439383 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:28.439755 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.440519 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.441789 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.442154 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.442234 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:28.442649 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:28.442718 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:28.442766 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:28.442951 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 00:48:28.443011 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:48:28.443316 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:28.438229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54744 (local address 127.29.84.190:41493)
0806 00:48:28.438694 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:28.438699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:28.438714 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:48:28.438805 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:28.438810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:28.438968 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:28.439123 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:28.439409 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.440107 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.441282 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:28.441699 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:28.442118 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:28.442248 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:28.442826 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:28.442935 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:28.443083 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 00:48:28.443150 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":79,"threads_started":1}
I20260806 00:48:28.450485 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 31531
I20260806 00:48:28.465471 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 31643
I20260806 00:48:28.474694 30034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdVRgL9/build/debug/bin/kudu with pid 31754
2026-08-06T00:48:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3296 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6551 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287644 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST